Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Target allocator TLS Implementation #239

Merged
merged 11 commits into from
Oct 9, 2024

Conversation

okankoAMZ
Copy link
Contributor

@okankoAMZ okankoAMZ commented Oct 2, 2024

Issue

Currently, Target Allocator(TA)'s Scrape Server is a http server. We want Target Allocator to be only accessed by the Agent; thus, we want the scrape server to be a https server.

Description of changes:

  • Update the server to use https server instead of http
  • Disabled http functionality
  • Update operator to mount the certificate volumes based on secrets.
  • Changed TA container's access port to 8443
  • Removed livez and readyz probes. (left them accessible from https server)

Warning

This PR Requires changes to the helm chart to be fully working, without them it will not crash or break; however, collector(agent) will not be able to ping the Target Allocator Container

Testing

Setup

For this testing since the endpoint can only be accessed through the agent I added a mock-collector which just pings these endpoint on repeat using the certification on the pod.

  1. make container && make container-push to generate the new operator container
  2. make ta-build-and-push to generate new Target Allocator container
  3. Edit helm-charts to use these new container images , also replaced the agent image with the mock-collector one.
  4. On a cluster with no amazon-cloudwatch-agent namespace deployed, deploy these changes with helm upgrade

Result

Pods

Pods

Service Description

Service Description

Mock-Collector Output After boot-up

Mock-Collector Output After boot-up

Target Allocator Output

Target Allocator Output

Note

I have removed that config line after testing

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@okankoAMZ okankoAMZ marked this pull request as ready for review October 2, 2024 16:50
versions.txt Outdated Show resolved Hide resolved
versions.txt Outdated Show resolved Hide resolved
@okankoAMZ okankoAMZ self-assigned this Oct 3, 2024
@okankoAMZ
Copy link
Contributor Author

okankoAMZ commented Oct 3, 2024

I'll rebase the commits to clean-it -up after the initial review

Makefile Show resolved Hide resolved
Makefile Show resolved Hide resolved
musa-asad
musa-asad previously approved these changes Oct 3, 2024
Copy link
Contributor

@mitali-salvi mitali-salvi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix for selectors for target-allocator is separate from the TLS functionality, I would prefer if its a separate PR altogether. Makes it easier to review PR and rollback/root-cause if there's an issue again

The selector label needs to be added to the target-allocator deployment as well to associate it with the service. This is how the cloudwatch-agent daemon-set does it. https://github.com/aws/amazon-cloudwatch-agent-operator/blob/main/internal/manifests/collector/daemonset.go#L31

internal/manifests/targetallocator/service.go Outdated Show resolved Hide resolved
internal/manifests/targetallocator/service.go Outdated Show resolved Hide resolved
internal/naming/main.go Outdated Show resolved Hide resolved
ItielOlenick and others added 10 commits October 7, 2024 17:47
* Added https server, tests, secret marshalling

* Added changelog

* Moved TLS config. Cleaned up server.

* TLS config as struct method. Go Idioms

* Removed default tls file paths. Minor cleanup.
@okankoAMZ okankoAMZ merged commit bfb6b3a into aws:target-allocator Oct 9, 2024
okankoAMZ added a commit that referenced this pull request Nov 26, 2024
* Implemented Target Allocator Container (#214)

* Merge `main` into `target-allocator` (#232)

NodeJS merging-in from main

* Supporting K8s 1.31 (#222) (#236)

Co-authored-by: Mitali Salvi <44349099+mitali-salvi@users.noreply.github.com>

* Implemented TargetAllocator resource deployments. (#208)

* Adding target-allocator label to service selectors (#242)

* Target allocator TLS Implementation  (#239)

* Ta https server (#2921)

* Added https server, tests, secret marshalling


---------

Co-authored-by: ItielOlenick <67790309+ItielOlenick@users.noreply.github.com>

* [Target Allocator] Enable Deployment and Daemonset modes for Agent (#253)

* Changes error to warning

* [CI/CD] Add Target Allocator(TA) Build to Build and Upload Workflow (#247)

* edited workflow

* Clean up  managed resources when disabled (#255)

* Reconciler now removes un-used managed resources for CWA collector

* remove pprof endpoint (#260)

* [TA] One service per Target Allocator  (#259)

* added one-service per TA

* Setup cert-watcher for TA server cert (#264)

* [TA] Target Allocator TLS Unit-tests (#265)

* TLS tests

* Injecting Prometheus path if not specified in agent config (#258)

* Injecting Prom path if it doesn't exist

* Rebasing Target Allocator Branch to Main  (#266)

* Adding support for NodeJS auto instrumentation and integ tests (#220)

* Support configurable resources for NodeJS. (#225)

* Supporting JMX annotations (#240)

* Add support for a supplemental YAML configuration for the CloudWatchAgent (#241)

* Changed naming for OTLP container ports from agent JSON (#252)

* Updated Release Notes for 1.8.0 (#251)

* Adjust EKS add-on integration test service count expectations (#256)

* Add integration tests for JMX. (#250)

* Implemented Target Allocator Container (#214)

* Implemented TargetAllocator resource deployments. (#208)

* Update cmd/amazon-cloudwatch-agent-target-allocator/config/config.go

Co-authored-by: Musa <musaasad@amazon.com>

* Update internal/config/main.go

Co-authored-by: Musa <musaasad@amazon.com>

---------

Co-authored-by: Parampreet Singh <50599809+Paramadon@users.noreply.github.com>
Co-authored-by: Musa <musaasad@amazon.com>
Co-authored-by: Mitali Salvi <44349099+mitali-salvi@users.noreply.github.com>
Co-authored-by: Jeffrey Chien <chienjef@amazon.com>


---------

Co-authored-by: Musa <musaasad@amazon.com>
Co-authored-by: Mitali Salvi <44349099+mitali-salvi@users.noreply.github.com>
Co-authored-by: ItielOlenick <67790309+ItielOlenick@users.noreply.github.com>
Co-authored-by: Kaushik Surya <108111936+sky333999@users.noreply.github.com>
Co-authored-by: Parampreet Singh <50599809+Paramadon@users.noreply.github.com>
Co-authored-by: Jeffrey Chien <chienjef@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants