Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prometheus configuration auto-reloading #355

Merged
merged 6 commits into from
Jun 10, 2024

Conversation

jb3
Copy link
Member

@jb3 jb3 commented Jun 9, 2024

I have created a repository, owl-corp/inotify-base, which is a
Debian slim image with some added niceities to trigger a script run on changes
to a given path.

We then have a script in this repository to reload Prometheus using the
management API and notify in the Discord channel if this has succeeded or not.

I have also enabled scraping of the Prometheus server and also added an alert
for failed config reloads to integrate it into our existing alerting pipeline.

Makes progress on #266, there is still no auto-deployment of content from this repository.

@jb3
Copy link
Member Author

jb3 commented Jun 9, 2024

image

@jb3 jb3 added component: networking An issue relating to a host networking (e.g. DNS, WireGuard, SSH) component: monitoring An issue relating to a monitoring component (e.g. Prometheus, Grafana) webscale group: kubernetes Issues and pull requests related to the Kubernetes setup labels Jun 9, 2024
@jchristgit
Copy link
Member

jchristgit commented Jun 10, 2024 via email

@jb3 jb3 force-pushed the jb3/kubernetes/266-prometheus-config-reload branch from 3f2e65d to 990cefd Compare June 10, 2024 13:14
@jb3
Copy link
Member Author

jb3 commented Jun 10, 2024

All review comments have been addressed in the force push.

@jb3 jb3 requested a review from jchristgit June 10, 2024 13:15
@jb3 jb3 force-pushed the jb3/kubernetes/266-prometheus-config-reload branch from 990cefd to 499bb71 Compare June 10, 2024 13:24
@jb3 jb3 force-pushed the jb3/kubernetes/266-prometheus-config-reload branch from 499bb71 to 7e76e2a Compare June 10, 2024 13:33
@jchristgit jchristgit merged commit 3b4613e into main Jun 10, 2024
8 checks passed
@jchristgit jchristgit deleted the jb3/kubernetes/266-prometheus-config-reload branch June 10, 2024 15:36
@jchristgit
Copy link
Member

Thanks!

1 similar comment
Copy link

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: monitoring An issue relating to a monitoring component (e.g. Prometheus, Grafana) component: networking An issue relating to a host networking (e.g. DNS, WireGuard, SSH) group: kubernetes Issues and pull requests related to the Kubernetes setup webscale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants