Emit more histogram buckets for cortex-querier_storegateway_refetches… #6570
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SungJin1212
force-pushed
the
Emit-more-buckets-for-cortex-querier_storegateway_refetches_per_query
branch
4 times, most recently
from
b50d72c to
e50e81d
Compare
yeya24
reviewed
Feb 2, 2025
| @@ -123,7 +123,7 @@ func newBlocksStoreQueryableMetrics(reg prometheus.Registerer) *blocksStoreQuery | |||
| Namespace: "cortex", | |||
| Name: "querier_storegateway_refetches_per_query", | |||
| Help: "Number of re-fetches attempted while querying store-gateway instances due to missing blocks.", | |||
| Buckets: []float64{0, 1, 2}, | |||
| Buckets: []float64{0, 1, 2, 4, 8, 16}, | |||
There was a problem hiding this comment.
We have a customized value of 5 retries. I don't know if it makes sense to have 16 retries. Shall we keep upper limit to 8?
There was a problem hiding this comment.
Can't we retry as many times as the user specifies? (https://github.com/cortexproject/cortex/blob/master/pkg/querier/blocks_store_queryable.go#L542)
There was a problem hiding this comment.
That's right. But I don't think we should recommend users to retry up to 16 times. I mean in general 3 to 7 retries make the most of sense. But maybe that's our usecase.
Do you have a usecase of using that many retries?
There was a problem hiding this comment.
Oh, you're right. let's fix it.
SungJin1212
force-pushed
the
Emit-more-buckets-for-cortex-querier_storegateway_refetches_per_query
branch
from
e50e81d to
b3d3e16
Compare
SungJin1212
force-pushed
the
Emit-more-buckets-for-cortex-querier_storegateway_refetches_per_query
branch
2 times, most recently
from
4880120 to
e85f1f9
Compare
yeya24
reviewed
Feb 4, 2025
…_per_query Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>
SungJin1212
force-pushed
the
Emit-more-buckets-for-cortex-querier_storegateway_refetches_per_query
branch
from
e85f1f9 to
ed3a42a
Compare
yeya24
approved these changes
Feb 6, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The #6276 exposes consistency check attempts as a flag; this PR emits more histogram buckets on the
cortex-querier_storegateway_refetches_per_querymetric.Which issue(s) this PR fixes:
Fixes #
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]