Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Driver/Input: Migrate audio backend to Symphonia (#89)
* Initial Symphonia libopus wrapper * Messy DCA(1) framing for Symphonia, DCA export on `Compressed` Export is mainly for testing, but I think it'd also be a nice backport. * Primitive mixer code. * Early measurements, thinking about right granularity for resampling wrt. different input packet sizes. * Some structuring, committing for now. * Prelim integration of mix code (not input readying yet...) * Minor progress on dropping in new inputs... * Deps: Depend on Serenity's `next` branch on Songbird's `next` branch (#6) * Depend on Serenity's `next` branch on Songbird's `next` branch * Update the examples to use the `next` branch * Deps: Bump streamcatcher version -> 1.0 (#93) * Examples: Fix serenity-next cache accesses (#99) Serenity's cache design has changed, this (finally) prevents the examples from failing to compile on CI. * Gateway: Add generics to `Call` methods. (#102) Adds generics to any `Id` types on `Call`. Also includes the overlooked `Songbird::get_or_insert`. Closes #94. Tested using `cargo make ready`. * Gateway: Remove lifetime from Serenity setup trait (#103) This matches a recent serenity change where `ClientBuilder` no longer has an explicit lifetime parameter. This was tested using `cargo make ready`. * Working MP3 playback. MP3s now work great under the convert -> resample -> mix pipeline, even across unclean frame boundaries. Main issue seemed to be the number of internal subchunks in the resampler, which is still totally opaque... Oh well. Actual instantiation of Lazy->Wrapped and Wrapped->Parsed elements in the driver is not yet covered. Other formats are broken due to handling of the default track id, which is currently a) messy, and b) incorrect. * Dumb hack for Ogg playback. Oggs have some frames whose length is 0, yet must be decoded. I assume this covers the entire coder delay. * Lazy input creation pipeline. * Use new `last_decoded` Decoder semantics. * Lazy inputs work as expected -- for Files at least. * Http lazy inputs! Tested with a URL extracted from bandcamp via youtube-dl, also includes the scaffolding necessary to have Reads/Seeks pass between sync/async boundaries. This format is MP3, streamed in over an HTTP request via reqwest. Seeks not tested with e.g. Async Files -- they are programmed, though. * Seek support. Moves all sync creation/parsing/seeking etc. over to an elastically-sized thread pool. Since basically everything is now a restartable, this means that the `ForwardOnly` distinction can be handled really cleanly in general and we can try to recreate sources as needed. * Basic youtube-dl input, remove old tokio support. Asks for any streams without webm, since they're likely to be golden right now. * Update to 2021 edition * Re-enable SetTrack * Redesign cached::Memory, port cached::Compressed * Opus frame passthrough support restored * Stereo/mono mix target support. * DCA seek support, metadata fixes. * WIP reorganisation, dead code removal. * Deps: Depend on Serenity's `next` branch on Songbird's `next` branch (#6) * Depend on Serenity's `next` branch on Songbird's `next` branch * Update the examples to use the `next` branch * Deps: Bump streamcatcher version -> 1.0 (#93) * Examples: Fix serenity-next cache accesses (#99) Serenity's cache design has changed, this (finally) prevents the examples from failing to compile on CI. * Gateway: Add generics to `Call` methods. (#102) Adds generics to any `Id` types on `Call`. Also includes the overlooked `Songbird::get_or_insert`. Closes #94. Tested using `cargo make ready`. * Gateway: Remove lifetime from Serenity setup trait (#103) This matches a recent serenity change where `ClientBuilder` no longer has an explicit lifetime parameter. This was tested using `cargo make ready`. * The great clippification. * Gateway: Generic Shard and Twilight v0.8 Support (#109) This PR adds support for twilight v0.8, mainly adapting to significant API changes introduced by v0.7. As a result of these, twilight no longer accepts arbitrary JSON input, so it seemed sensible to adapt our `Shard` design to no longer require the same. Adding to this, I've added in a trait to allow an arbitrary `Shard` to be installed, given only an implementation of a method to send a `VoiceStateUpdate`. Together, `Sharder::Generic` (songbird::shards::VoiceUpdate) and `Shard::Generic` (songbird::shards::GenericSharder) should allow any library to be hooked in to Songbird. This PR was tested using `cargo make ready` and by manually testing `examples/twilight`. * Gateway: Twilight v0.9 support (#110) This handles twilight's migration to a unified `Id` type, which is the only design change needing any handling on our part. All our `From`/`Into`s are covered now, and deprecated type aliases are no longer used. This was tested using `cargo make ready` and by manually running "examples/twilight". * Update Audiopus -> 0.3.0-rc.0. Simple enough, mainly forces us to do all the buffer size checks locally instead. * WebM support merged upstream! Allow WebM from youtube-dl. FFmpeg is finally gone. * Fix Symphonia to v0.5 This version includes all the API changes we really need to function, and adds MKV support to a satisfactory level. This should free this PR up for an actual release down the line. Additionally, this changes the symphonia dependency to remove default features so that choice over the codec suite (barring opus) is entirely user-driven. * Catching up on documentation. Also some assorted clippy niceties. * Update ARCHITECTURE.md * Add (untested) wrapper for raw PCM data. * Add cached::decompressed, fix interleaved PCM adapter, nice docs for Input. * Fix non-Send future in TrackQueue::add_raw This arose from metadata collection occurring in this function. * Resume HTTP sessions on failure. Youtube-dl streams should now behave much, much nicer. * Clean up imports in the HTTP/async adapter. * Deps: Depend on Serenity's `next` branch on Songbird's `next` branch (#6) * Depend on Serenity's `next` branch on Songbird's `next` branch * Update the examples to use the `next` branch * Deps: Bump streamcatcher version -> 1.0 (#93) * Examples: Fix serenity-next cache accesses (#99) Serenity's cache design has changed, this (finally) prevents the examples from failing to compile on CI. * Gateway: Add generics to `Call` methods. (#102) Adds generics to any `Id` types on `Call`. Also includes the overlooked `Songbird::get_or_insert`. Closes #94. Tested using `cargo make ready`. * Gateway: Remove lifetime from Serenity setup trait (#103) This matches a recent serenity change where `ClientBuilder` no longer has an explicit lifetime parameter. This was tested using `cargo make ready`. * Gateway: Generic Shard and Twilight v0.8 Support (#109) This PR adds support for twilight v0.8, mainly adapting to significant API changes introduced by v0.7. As a result of these, twilight no longer accepts arbitrary JSON input, so it seemed sensible to adapt our `Shard` design to no longer require the same. Adding to this, I've added in a trait to allow an arbitrary `Shard` to be installed, given only an implementation of a method to send a `VoiceStateUpdate`. Together, `Sharder::Generic` (songbird::shards::VoiceUpdate) and `Shard::Generic` (songbird::shards::GenericSharder) should allow any library to be hooked in to Songbird. This PR was tested using `cargo make ready` and by manually testing `examples/twilight`. * Gateway: Twilight v0.9 support (#110) This handles twilight's migration to a unified `Id` type, which is the only design change needing any handling on our part. All our `From`/`Into`s are covered now, and deprecated type aliases are no longer used. This was tested using `cargo make ready` and by manually running "examples/twilight". * Deps: Depend on Serenity's `next` branch on Songbird's `next` branch (#6) * Depend on Serenity's `next` branch on Songbird's `next` branch * Update the examples to use the `next` branch * Deps: Bump streamcatcher version -> 1.0 (#93) * Examples: Fix serenity-next cache accesses (#99) Serenity's cache design has changed, this (finally) prevents the examples from failing to compile on CI. * Gateway: Add generics to `Call` methods. (#102) Adds generics to any `Id` types on `Call`. Also includes the overlooked `Songbird::get_or_insert`. Closes #94. Tested using `cargo make ready`. * Gateway: Remove lifetime from Serenity setup trait (#103) This matches a recent serenity change where `ClientBuilder` no longer has an explicit lifetime parameter. This was tested using `cargo make ready`. * Gateway: Generic Shard and Twilight v0.8 Support (#109) This PR adds support for twilight v0.8, mainly adapting to significant API changes introduced by v0.7. As a result of these, twilight no longer accepts arbitrary JSON input, so it seemed sensible to adapt our `Shard` design to no longer require the same. Adding to this, I've added in a trait to allow an arbitrary `Shard` to be installed, given only an implementation of a method to send a `VoiceStateUpdate`. Together, `Sharder::Generic` (songbird::shards::VoiceUpdate) and `Shard::Generic` (songbird::shards::GenericSharder) should allow any library to be hooked in to Songbird. This PR was tested using `cargo make ready` and by manually testing `examples/twilight`. * Gateway: Twilight v0.9 support (#110) This handles twilight's migration to a unified `Id` type, which is the only design change needing any handling on our part. All our `From`/`Into`s are covered now, and deprecated type aliases are no longer used. This was tested using `cargo make ready` and by manually running "examples/twilight". * Events: Remove deprecated events. (#115) This removes the `ClientConnect`, `DriverConnectFailed`, `DriverReconnectFailed` and `SsrcKnown` events. Tested using `cargo make ready`. * Example cleanup. * Update benchmarks to use symphonia, fix Compressed with 48kHz input Hopefully, this should make it easier to see whether internal restructuring of Tracks has any odd effects. The benches are messy, but they function at least. * Some restructuring work on Tracks More to be done, just need to perf-test AoS vs SoA now. * driver, queue: return track handle when adding an `Input` to the queue (#116) * Gateway: Twilight v0.10 support (#117) * Update to rubato v0.11. * Driver, Gateway: Remove tokio 0.2 support (#118) * Remove tokio 0.2 compat * Remove tokio 0.2 test * Remove tokio 0.2 CI * Deps: Bump dependencies and document bumped MSRV (#119) * Bump dependencies * Document bumped MSRV * Driver: Prevent panic when decrypting undersized RTP packets (#122) Decrypt logic had two locations where the nonce would be separated from the payload without verifying the buffer size first, causing a panic for small packets. Nonce and header removal now return an error if there are insufficient bytes. Tested using `cargo make ready`, with some new tests to check that small packets simply return an `Err(...)`, and that encryption/decryption still function. * Accidentally an await. * Driver: Remove spin_sleep in `Mixer::march_deadline` (#124) * Remove spin_sleep * Remove comment * Examples: support new Serenity Intents init Fixes up the serenity examples to account for a bunch of API changes on `next`/v0.11. Tested using `cargo make ready`. * Deps: Update to Audiopus v0.3.0-rc.0 (#125) Tested using `cargo make ready`. Co-authored-by: André Vennberg <andre.vennberg@gmail.com> * Update deps * Fix otherwise broken state-change commands. * Try fix clippy warnings * fmt * Bump MSRV to 1.61.0. * Into<Input> for static/owned bytes * Update twilight support to twilight 0.11 (#132) * update twilight support to twilight 0.11 * rustfmt * Fix broken 22.05kHz->48kHz resample Doubles the resample chunk length to 10ms: I assume something odd is going on w/ integer math in rubato, but I'm not really willing to look into that * Tracks + TrackHandles rework. `create_player` and related functions are gone, Handles are now created *when* a Track is actually handed over. `Track`s are now very small, since most state isn't actually needed/generated until you reach the driver. This means we can avoid having to make a bunch of `Track` state private to prevent users from, e.g., taking illegal actions via a `TrackHandle`. To avoid exposing internal details, `TrackHandle::action` now gets a `View` of track state generated as needed. Still missing some docs and some cleanup. Still missing actual access to metadata, but it will happen through `View`. * Add `aux_metadata()` to `Input`. This should make it easier to access auxiliary metadata after, e.g., conversion to a `Track`. * Expose probe and format metadata via `action`. * Add + Expose ReadyState via View. It's now possible to view whether a track is playable, lazy, or mid-preparation. * Helper methods for input upgrading/creation/created access. * Temporarily pin serenity version. I don't have the time to rewrite examples to work around the Send nightmares introduced with the dashmap refs change atm: moreso if this is being released to target 0.11 * Create + Parse error signalling, listeners for TrackErrors * Decode error handling + events. * Some cleanup in the mixer logic. * Refactor and reorganise mixing internals. * Code restructure for `crate::input`. Should be cleaner and easier to navigate now, external API is still the same (and pretty flat, hierarchically). * Document `Symphonia` requirement in README and docs root * Accidentally a space * Update mixing benchmarks wrt. restructured code * Document some errors, explain how to get (Aux)Metadata. * Clarify failure reasons for Input::aux_metadata * Document remaining types. * A slightly exhausting example on the wonders of `Input` * Removing some old error variants. * Make Metadata link to action clearer. * Make Passthrough a bit more lenient A good few test inputs had final frames shorter than 20ms, meaning that they'd take an encoder delay's worth of silence before the last chunk of playback, which sounds awful. This moves Passthrough enforcement to a 'three strikes' system to give a little more leeway on this front, preventing really awkward and expensive silent bursts at rhe end of Opus tracks. * Appease Clippy * Attempt to add yt-dlp to CI, make it the preferred fork in README. * Clarify audiopus v0.3.0 build details. * Make Driver.add_raw a public function * Fix clippy::must_use_candidate * Fix clippy::default_trait_access * Fix clippy::match_same_arms * Fix clippy::semicolon_if_nothing_returned * Fix clippy::enum_glob_use * Fix clippy::return_self_not_must_use * Fix clippy::unused_self * Fix clippy::trivially_copy_pass_by_ref * Fix clippy::let_underscore_drop * Fix clippy::map_unwrap_or * Fix clippy::if_not_else * Fix clippy::unnested_or_patterns * Fix clippy::single_match_else * Fmt and fix clippy::mut_mut * Fix clippy::similar_names * Fix clippy::needless_pass_by_value * Fix clippy::match_wildcard_for_single_variants * Fix clippy::redundant_closure_for_method_calls * Fix clippy::doc_markdown * Fix clippy::explicit_iter_loop * Swap warn specific to warn all and allow, fix last couple warns * Fix comment in lib.rs Co-authored-by: Kyle Simpson <kyleandrew.simpson@gmail.com> * Update DiscoRTP * Initial work on headless test config for driver * An actual driver test for the queue. * Pin to another 0.11-ish version of next * From<ChildContainer> for Input * Recursive From/Into for all Input sources -> Track * De-pub TrackHandle::new This was definitely exposing too many implementation details. * Typos in README/lib-docs * Remove reused code from `Decompressed`'s byte transform * Revisit LiveInput::promote Slightly cleaner. Failure to find a track (or to create a decoder matching the format's default track) now result in an error rather than a panic. * Placeholder YTDL test, convenience methods for output checking * Add missing `Config` fields. * Add Preparing and Playable events, expose ReadyState on TrackState. This simplifies (and makes consistent) a *lot* of test methods. * YoutubeDl seek tests. * Fix: `Err(_)` events now also fire `End` Fixes an issue where an errored track on the queue would just block (i.e., in event of a 404). Later down the line, some errors may not fire ended (i.e., decode errors) which may be configured to play on. * Move test/example resources to unified location. * Hopefully fix windows CI. Unsure what's up with Http/Opus seeking. * Add timeouts to all tests, unify code paths for format checking. Seems to be the best way to rat out driver panics. * Document/explain safety of OpusDecoder. * Manually check for backseeking with one-way Inputs It seems that some formats (MP4, Ogg) don't have logic to check that you're about to make a Very Bad Decision. We need to cache the inner bytestream's `is_seekable` and check this against a backwards seek ourselves, since pulling it out of a constructed source is nigh-on impossible. MKV/WebM seek bug is still present: so far as I can tell it's because it uses a very different time base in the format than the decoder instances. I need to confirm whether other codecs do any local time_base corrections. * Remove one incidental panic. * Update tests to remove reliance on racy behaviour, record thoughts on MKV timing hell. * Temporary pin to symphonia PR to keep the CI ticking. * A nightly clippy nit. * Temp Symphonia branch for my pile of fixes. * Rework `add_raw` into `add_`/`enqueue_with_preload`. This does a little bit of internal reworking to make this feasible, but this prevents us from having one function on `TrackHandles` while the rest are on `Track`s (so more user-friendly). Similarly, this prevents us from having to impose conditions on the target `TrackHandle` -- "you must pause your track first" is a bit of a footgun. This also applies event listeners *before* tracks are handed over to the driver, which is more correct and means we don't need to wait a tick or similar. * Next got forced, again. * Rework ClientData to wrap in Option instead of initialised bool (#2) * Repair some broken doc-links. * Add cfg(test) around all testing paths (#5) * Properly use cfg(test) * Use Packet * Reduce default thread pool allocation. * Fix packet sending outside of cfg(test) * Use YoutubeDl-suggested header values in HttpRequest This should hopefully get past the ratelimiting. * Support serenity@next instead of pinning version (#7) * Support serenity@next instead of pinning version * Fix tests and re-add From * Update examples to work with serenity@next. Also, twilight example was a bit broken. * Bypass Youtube ratelimiting Youtube expects a very particular set of HTTP headers, otherwise it will allocate you a very snug 70kB/s download rate. This didn't impact audio playout too badly, but it did prevent seeking as fast as required. Seeks now work just as well on Youtube as they do for all other HTTP sources. * Seek/MakePlayable callbacks on TrackHandle. * Unbreak tests. * Callback tests, fix input ready notification on paused tracks. * Document TrackCallback * TrackHandle::seek_time -> seek * Prevent track times from ticking while preparing. Also, make an in-seek track return to `Preparing`. * Accidental commit of some seek testing in example. * Tests ensuring Loops work in perpetuity. * Fix looping on streamed MKVs Seeking back to t=0 requires a page realignment on Youtube's WebMs, which is impossible (backseek of around 7 bytes) and fails due to existing create-loop prevention checks. Seeks to zero are now ignored after re-creating a source, as the track can be safely assumed to be at its beginning. * Fix HTTP Range resume Range requests were being horribly broken by a double range field that reqwest didn't filter out. Oops! * Some leniency on tougher tests. * Remove unnecessary poison messages (#6) * Remove unnecessary poison messages * Re-add Poison for CoreMessage * Use OnceCell and DashMap (#8) * Format, alphabetise feature deps. * Fix ShardMessage for serenity::next * Structify YoutubeDL Json reading * Make `url` field mandatory in YtDL Output. * FFprobe output as File::aux_metadata * Quick once-over of the serenity examples. * Review Pt. 1: Doc mix logic, extend passthrough to one *live* track This is most relevant for queue users -- an extra track would cause opus passthrough to end, even though only one track was being actively used in mixing. * Some unneeded refs caught by clippy Corollary: this same new clippy lint adds a ton of false positives which *will* fail to compile. * Minor cleanup: not a fan of From<&Type> vs. dedicated method. * Track removals from event context, sans `Vec` * Review Pt 2: Mixer task. * Act on queued seeks given mid-track--prepare. * Review pt 3: Starting on inputs Will leave 're-computing filesize guesses' to a future issue. * Remove unnecessary chrono dep (#9) * Adapt examples to new serenity framework. * impl Error for Cached adapters * Review pt 4: adapters. * Review pt 5: Inputs * Finis Co-authored-by: Alex M. M <acdenissk69@gmail.com> Co-authored-by: tkt <37575408+tktcorporation@users.noreply.github.com> Co-authored-by: Jan <66554238+vaporox@users.noreply.github.com> Co-authored-by: Gnome! <45660393+GnomedDev@users.noreply.github.com> Co-authored-by: André Vennberg <andre.vennberg@gmail.com> Co-authored-by: Gnome <david2005thomas@gmail.com> Co-authored-by: Erk <Erk-@users.noreply.github.com> Co-authored-by: Alakh <36898190+alakhpc@users.noreply.github.com>
- Loading branch information
9 people
authored
Jul 23, 2022
1 parent
0d66467
commit 5547de2