Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cloudwatch-agent pods has prometheus annotations but connection is refused #190

Open
henkka opened this issue Jul 3, 2024 · 2 comments
Open

Comments

@henkka
Copy link

henkka commented Jul 3, 2024

Issue Description:

The cloudwatch-agent pods launched by the DaemonSet/cloudwatch-agent created by this operator contain the following annotations:

  • prometheus.io/scrape: true
  • prometheus.io/port: 8888
  • prometheus.io/path: /metrics

However, when attempting to connect to :8888/metrics, a "connection refused" error occurs.

Request for Enhancement:

  1. Expose Prometheus Metrics: Is it possible to configure cloudwatch-agent to expose Prometheus metrics?
  2. Configurable Annotations: If the cloudwatch-agent can be configured to expose Prometheus metrics, please make this behavior configurable. The pod annotations should be added only if this feature is enabled. Otherwise, these annotations should not be added to avoid confusion and potential connection errors.

https://github.com/aws/amazon-cloudwatch-agent-operator/blame/main/internal/manifests/collector/annotations.go#L18-L21

@mitali-salvi mitali-salvi added enhancement New feature or request and removed enhancement New feature or request labels Jul 31, 2024
@lisguo
Copy link
Contributor

lisguo commented Aug 12, 2024

Hello, we are aware of a feature gap with prometheus. There is a related github issue in the helm chart: aws-observability/helm-charts#55

@krisko
Copy link

krisko commented Aug 28, 2024

Hello, we are aware of a feature gap with prometheus. There is a related github issue in the helm chart: aws-observability/helm-charts#55

Hello @lisguo, I think that this issue is not about being able to scrape prometheus metrics with the agent. It's about that the agent has prometheus annotations and should expose metrics via metrics endpoint. But that endpoint is not available. Because of that we are getting alerts about unreachable metrics endpoint on the agent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants