-
Notifications
You must be signed in to change notification settings - Fork 793
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compaction is Not Running Properly Because of Out-of-order Chunk #5584
Comments
@LERUfic May I know which version of Prometheus are you using? I am wondering if it is the same issue as thanos-io/thanos#6723 |
sure @yeya24
PS: opss wrong account |
This seems like a bug in Thanos https://github.com/thanos-io/thanos/blob/main/pkg/compact/compact.go#L1394. I will create an issue on Thanos side. |
I see thank you for the response. For now I marked the chunks with
And seems like there's no error for now but the compaction still has not finished yet. We will monitor this for a while. |
Describe the bug
Due to out-of-order chunk, the compactor is not performing compaction as expected. Although I have added the
skip_blocks_with_out_of_order_chunks_enabled: true
configuration, the block is not being marked as non-compact.To Reproduce
Steps to reproduce the behavior:
Expected behavior
I expect the compaction process is running smoothly and even the out-of-order happening it's should be skipped because of the
skip_blocks_with_out_of_order_chunks_enabled: true
config.Environment:
Additional Context
My compactor config
Runtime Config
Metrics
Error logs
The text was updated successfully, but these errors were encountered: