feat(quotas): Rate limit metrics before aggregator [INGEST-1655] #1540
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- we're seeing a decent amount of replay event parse failures in our outcomes table. We're not sure why exactly, so adding a warning here if we fail to parse the replay event. #skip-changelog closes getsentry/replay-backend#168
…te-limit-before-agg
jjbayer
changed the base branch from
master
to
feat/indexed-transactions-limiting
…te-limit-before-agg
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
In #1515 we implemented rate limiting functionality for metrics buckets, which are applied after flushing them from the metrics aggregator by checking redis.
This PR
When a quota is already exhausted, and that information has already been cached on the project state, there is no need to go through the aggregator at all. Instead, we apply the rate limiting logic before sending metrics or buckets into the aggregator.
Implementation details
The utility type
BucketLimiterwas converted to a genericMetricsLimiter<T>, whereTcan be eitherMetricorBucket.Notes
Will be merged into #1537 & deployed together.
#skip-changelog (there's already a line about rate limiting metrics buckets)