-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Thanos receive - failed to handle request, internal server error #7703
Comments
Hey @ALX-TH! I'm here to help you with the issue you're facing. Let's work together to get this sorted out. Prometheus fails to write chunks to Thanos receive starting from Thanos version 0.30.0 due to a change introduced in the following pull request:
This change likely altered the behavior of the remote write path, which could be causing the issue with Prometheus writing chunks to Thanos receive [1]. To address this issue, you may need to review and adjust your configuration to align with the changes introduced in version 0.30.0. Specifically, ensure that your configuration accounts for the new features and fixes:
Additionally, the expected HTTP status codes and error messages for the
Reviewing these changes and ensuring your configuration is updated accordingly should help resolve the issue. |
Hi all. I have an issue with thanos receive. Starting from thanos version 0.30.0 my prometheus server cannot write chunks to receiver. But at same time, if i using thanos 0.29.0 (and lower) - no issues. May u please help with solving problem ?
I suppose, something changed with Prometheus implementation (API ?) at thanos starting from version 0.30.0
Object Storage Provider:
GCS (Google Bucket)
What happened:
Prometheus cannot write chunks into thanos receive
What you expected to happen:
Prometheus should successfully write chunks into thanos receive
Full logs to relevant components:
Prometheus Logs
Thanos Logs
Configs:
Prometheus:
Thanos
Environment:
The text was updated successfully, but these errors were encountered: