Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add prometheus scrape jobs for all units #264

Merged
merged 18 commits into from
Jun 26, 2024
Merged

add prometheus scrape jobs for all units #264

merged 18 commits into from
Jun 26, 2024

Conversation

ca-scribner
Copy link
Contributor

@ca-scribner ca-scribner commented Jun 10, 2024

Issue

closes #151

Solution

Extends AlertmanagerCharm.self_scraping_job() to include all units, not just the leader, for metrics scraping.

Testing Instructions

juju deploy prometheus-k8s --channel edge --trust
charmcraft pack
juju deploy ./*.charm --resource alertmanager-image=ubuntu/alertmanager:0-22.04 --trust
juju relate alertmanager-k8s:self-metrics-endpoint prometheus-k8s

# wait for things to settle

jhack show-relation alertmanager-k8s prometheus-k8s
# Should see a single target provided

juju scale-application alertmanager-k8s 3
# wait for things to settle
jhack show-relation alertmanager-k8s prometheus-k8s
# Should see a target for each unit

# Check if prometheus has the targets
curl $PROMETHEUS_IP:9090/api/v1/targets \
  | jq '.data.activeTargets | .[] | {scrapeUrl, health}'
# should see a target for each am unit (plus one for prometheus)

Upgrade Notes

Upgrade should happen seemlessly. Nothing has changed to affect upgrades

This moves some logic for parsing peer addresses for WorkloadManager from the charm to WorkloadManager.  The reason for this is that the charm will be extended to use peer addresses for other configurations (metrics), and this makes it easier to add that functionality.
Moves logic deciding whether the charm's address uses http or https to a reusable location.  This was done to reuse this logic elsewhere in the charm.
Previously, AlertmanagerCharm provided metrics only for the leader.  This updates AlertmanagerCharm.self_scraping_job to generate a jobs list for MetricsEndpointProvider that includes all units, not just the leader.

Closes #151
@ca-scribner ca-scribner marked this pull request as ready for review June 11, 2024 20:44
@ca-scribner ca-scribner requested a review from a team as a code owner June 11, 2024 20:44
This uses target expansion provided by the prometheus charm that removes the need to explicitly define each target.  The downside of this is that they're rendered as IPs rather than FQDNs, which does not support TLS for the scrape
ca-scribner and others added 5 commits June 17, 2024 13:58
@ca-scribner ca-scribner merged commit 71c422f into main Jun 26, 2024
13 checks passed
@ca-scribner ca-scribner deleted the openg-1360 branch June 26, 2024 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Only leader scrape jobs are sent to Prometheus
3 participants