cluster: Jitter at the end of ntp reconcile loop #24726
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`reconcile_ntp_fiber` is spawned into the background potentially many many times concurrently. Because of how `process_delta` and friends call `reconcile_ntp_fiber` they mark the `wakeup_event` ready repeatedly so effectively all waiters will be woken up in one tick. This causes a massive task queue. Avoid this by making `reconcile_ntp_fiber` only at the end of the loop. This means that all invocations will have to make it through the `_reconciliation_sem` semaphore once first (which is concurrency limited).
CI test resultstest results on build#60413
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
reconcile_ntp_fiberis spawned into the background potentially manymany times concurrently.
Because of how
process_deltaand friends callreconcile_ntp_fiberthey mark the
wakeup_eventready repeatedly so effectively all waiterswill be woken up in one tick. This causes a massive task queue.
Avoid this by making
reconcile_ntp_fiberonly at the end of the loop.This means that all invocations will have to make it through the
_reconciliation_semsemaphore once first (which is concurrencylimited).
Backports Required
Release Notes