scx_layered: Implement empty LLC draining #1092
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Use < instead of <= when comparing against xllc_mig_min_ns so that 0 can disable it completely.
This is trivial to count from BPF but we'll also add per-LLC counts, so let's just do whatever we can do from userspace in userspace.
A layer is served by per-LLC DSQs. Only tasks that can be serviced by all CPUs in an LLC are put in the DSQ, so as long as a CPU is assigned to the layer in the LLC, forward progress is guaranteed. However, if a layer-LLC loses all its CPUs, there is no forward progress guarantee ignoring the antistall mechanism. For a confined layer, no CPU will be visiting the empty LLCs and even for a grouped or open layer, execution from an LLC without CPUs assigned is lower priority than owned execution and can easily starve. To resolve the problem, implement LLC draining mechanism. When layer-LLC loses all CPUs with tasks in it, draining is turned on and other CPUs assigned to the layer will alternate between their own execution and draining LLCs without any CPU. The interlockings between the involved code paths - refresh_cpumasks(), layered_enqueue() and layered_dispatch() - are rather intricate to guarantee that no tasks end up sitting in a CPU-less LLC. See comments for details.
etsal
approved these changes
Dec 10, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A layer is served by per-LLC DSQs. Only tasks that can be serviced by all
CPUs in an LLC are put in the DSQ, so as long as a CPU is assigned to the
layer in the LLC, forward progress is guaranteed. However, if a layer-LLC
loses all its CPUs, there is no forward progress guarantee ignoring the
antistall mechanism. For a confined layer, no CPU will be visiting the empty
LLCs and even for a grouped or open layer, execution from an LLC without
CPUs assigned is lower priority than owned execution and can easily starve.
To resolve the problem, implement LLC draining mechanism. When layer-LLC
loses all CPUs with tasks in it, draining is turned on and other CPUs
assigned to the layer will alternate between their own execution and
draining LLCs without any CPU. The interlockings between the involved code
paths - refresh_cpumasks(), layered_enqueue() and layered_dispatch() - are
rather intricate to guarantee that no tasks end up sitting in a CPU-less
LLC. See comments for details.