Driver: Implement audio scheduler #179
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
These are 25% faster but currently rely on unsafe, it remains to be seen whether the safe variants are as fast.
No perf loss so far as I can see
FelixMcFelix
force-pushed
the
mix-scheduler
branch
from
8be666e
to
dc42557
Compare
Test forthcoming...
FelixMcFelix
force-pushed
the
mix-scheduler
branch
from
6227b93
to
da02640
Compare
FelixMcFelix
added
enhancement
Not *actually* implemented the instrumentation, yet.
FelixMcFelix
force-pushed
the
mix-scheduler
branch
from
830bee5
to
f6e0cca
Compare
FelixMcFelix
force-pushed
the
mix-scheduler
branch
from
0f0d70d
to
af4c457
Compare
This should fix the one test which keeps breaking on GH runners.
This seemed to provide a decent speedup (4--10%) for multiple mixers, both float and opus.
FelixMcFelix
force-pushed
the
mix-scheduler
branch
from
7113125
to
c415f6b
Compare
FelixMcFelix
added a commit
to FelixMcFelix/songbird
that referenced
this pull request
Nov 20, 2023
This PR implements a custom scheduler for audio threads, which reduces thread use and (often) memory consumption. To save threads and memory (e.g., packet buffer allocations), Songbird parks Mixer tasks which do not have any live Tracks. These are now all co-located on a single async 'Idle' task. This task is responsible for managing UDP keepalive messages for each task, maintaining event state, and executing any Mixer task messages. Whenever any message arrives which adds a `Track`, the mixer task is moved to a live thread. The Idle task inspects task counts and execution time on each thread, choosing the first live thread with room, and creating a new one if needed. Each live thread is responsible for running as many live mixers as it can in a single tick every 20ms: this currently defaults to 16 mixers per thread, but is user-configurable. A live thread also stores RTP packet blocks to be written into by each sub-task. Each live thread has a conservative limit of 18ms that it will aim to stay under: if all work takes longer than this, it will offload the task with the highest mixing cost once per tick onto another (possibly new) live worker thread.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a scheduler for audio threads: idle tasks are all run on a single background async task, while live audio threads are grouped up into full sync threads. Tasks are dynamically promoted and demoted based on whether they have any live audio tracks. This adds in some other niceties like preventing packet buffer reallocations, where possible. I intend to add reclamation of packet buffers, worker threads, and offloading expensive tasks to other threads.
This defaults to grouping 16 live calls per thread, mainly to prevent issues with users not compiling in release mode and/or using weaker machines. This can be configured during
Configcreation, and should be tuned for users at scale -- for instance, a passthrough-only workload can serve up to ~660 calls per thread on a Ryzen 5700X.To put this in context, if all calls are live, this defaults to a 16x reduction in the amount of threads used. If only 30% of calls are live, the remainder all go in a single async task, giving an asymptotic 53x reduction of thread count.
TODO: