-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Alert missing for bucket operation failures on the sidecar #3005
Comments
Hello 👋 Looks like there was no activity on this issue for last 30 days. |
I haven't tried 0.15.0 but I don't see anything in the changelog |
Seeing this issue too, is it intentional? |
@brokencode64 I don't think it is intentional. We can always ask @daixiang0, if the removal was intentional. @daixiang0 WDYT? @SerialVelocity In any case, if you think it is an alert that's needed or we are missing any other alerts, please feel free to contribute. I'd be happy to review and include it in the next release. |
It is not intentional but a mistake, @SerialVelocity thanks for catching it! |
Hello 👋 Looks like there was no activity on this issue for the last two months. |
Still valid. |
Thanos, Prometheus and Golang version used:
quay.io/prometheus/prometheus:v2.20.0
quay.io/thanos/thanos:v0.14.0
Object Storage Provider:
Scaleway S3 compatible object store
What happened:
The sidecar was failing to upload to the S3 (my fault) but no alerts were triggered.
What you expected to happen:
An alert to be triggered.
How to reproduce it (as minimally and precisely as possible):
Set up a network policy that blocks uploading
Full logs to relevant components:
Anything else we need to know:
It looks like this used to work by having an alert on
thanos_objstore_bucket_operation_failures_total
. I actually found the issue when I looked at the thanos sidecar dashboard.It looks like #2002 removed the alert. I think only ones with
_s3_
/_gcs_
in the name were meant to be removed?The text was updated successfully, but these errors were encountered: