Driver/Input: Migrate audio backend to Symphonia

.github/workflows/ci.yml

@@ -39,7 +39,6 @@ jobs:
           - Windows
           - driver only
           - gateway only
-          - legacy tokio
 
         include:
           - name: beta
@@ -75,6 +74,16 @@ jobs:
           sudo apt-get update
           sudo apt-get install -y libopus-dev
 
+      - name: Install yt-dlp (Unix)
+        if: runner.os != 'Windows'
+        run: |
+          sudo wget https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -O /usr/local/bin/yt-dlp
+          sudo chmod a+rx /usr/local/bin/yt-dlp
+
+      - name: Install yt-dlp (Windows)
+        if: runner.os == 'Windows'
+        run: choco install yt-dlp
+
       - name: Setup cache
         if: runner.os != 'macOS'
         uses: actions/cache@v2
@@ -175,9 +184,9 @@ jobs:
       - name: 'Build serenity/voice_receive'
         working-directory: examples
         run: cargo build -p voice_receive
-      - name: 'Build serenity/voice_storage'
+      - name: 'Build serenity/voice_cached_audio'
         working-directory: examples
-        run: cargo build -p voice_storage
+        run: cargo build -p voice_cached_audio
       - name: 'Build twilight'
         working-directory: examples
         run: cargo build -p twilight

ARCHITECTURE.md

@@ -29,12 +29,14 @@ Songbird's **driver** is a mixed sync/async system for running voice connections
 Audio processing remains synchronous for the following reasons:
 * Encryption, encoding, and mixing are compute bound tasks which cannot be subdivided cleanly by the Tokio executor. Having these block the scheduler's finite thread count has a significant impact on servicing other tasks.
 * `Read` and `Seek` are considerably more user-friendly to use, implement, and integrate than `AsyncRead`, `AsyncBufRead`, and `AsyncSeek`.
+* Symphonia implements all of its functionality based on synchronous I/O.
 
 ## Tasks
 Songbird subdivides voice connection handling into several long- and short-lived tasks.
 
 * **Core**: Handles and directs commands received from the driver. Responsible for connection/reconnection, and creates network tasks.
 * **Mixer**: Combines audio sources together, Opus encodes the result, and encrypts the built packets every 20ms. Responsible for handling track commands/state. ***Synchronous***.
+* **Thread Pool**: A dynamically sized thread-pool for I/O tasks. Creates lazy tracks using `Compose` if sync creation is needed, otherwise spawns a tokio task. Seek operations always go to the thread pool. ***Synchronous***.
 * **Disposer**: Used by mixer thread to dispose of data with potentially long/blocking `Drop` implementations (i.e., audio sources). ***Synchronous***.
 * **Events**: Stores and runs event handlers, tracks event timing, and handles 
 * **Websocket**: *Network task.* Sends speaking status updates and keepalives to Discord, and receives client (dis)connect events.
@@ -52,23 +54,22 @@ src/driver/*
 ## Audio handling
 
 ### Input
-Inputs are raw audio sources: composed of a `Reader` (which can be `Read`-only or `Read + Seek`), a framing mechanism, and a codec.
-Several wrappers exist to add `Seek` capabilities to one-way streams via storage or explicitly recreating the struct.
+Inputs are audio sources supporting lazy initialisation, being either:
+* **lazy inputs**—a trait object which allows an instructions to create an audio source to be cheaply stored. This will be initialised when needed either synchronously or asynchronously based on what which methods the trait object supports.
+* **live inputs**—a usable audio object implementing `MediaSource: Read + Seek`. `Seek` support may be dummied in, as seek use and support is gated by `MediaSource`. These can be passed in at various stages of processing by symphonia.
 
-Framing is not always needed (`Raw`), but makes it possible to consume the correct number of bytes needed to decode one audio packet (and/or simplify skipping through the stream).
-Currently, Opus and raw (`i16`/`f32`) audio sources are supported, though only the DCA framing for Opus is implemented.
-At present, the use of the FFmpeg executable allows us to receive raw input, but at heavy memory cost.
-Further implementations are possible in the present framework (e.g., WebM/MKV and Ogg containers, MP3 and linked FFI FFmpeg as codecs).
+Several wrappers exist to add `Seek` capabilities to one-way streams via storage or explicitly recreating the struct, `AsyncRead` adapters, and raw audio input adapters.
 
 Internally, the mixer uses floating-point audio to prevent clipping and allow more granular volume control.
-If a source is known to use the Opus codec (and is the only source), then it can bypass mixing altogether.
+Symphonia is used to demux and decode input files in a variety of formats into this floating-point buffer: songbird supports all codecs and containers which are part of the symphonia project, while adding support for Opus decoding and DCA1 container files.
+If a source uses the Opus codec (and is the only source), then it can bypass mixing and re-encoding altogether, saving CPU cycles per server.
 
 ```
 src/input/*
 ```
 
 ### Tracks
-Tracks hold additional state which is expected to change over the lifetime of  a track: position, play state, and modifiers like volume.
+Tracks hold additional state which is expected to change over the lifetime of a track: position, play state, and modifiers like volume.
 Tracks (and their handles) also allow per-source events to be inserted.
 
 Tracks are defined in user code, where they are fully modifiable, before being passed into the driver.

Cargo.toml

@@ -2,23 +2,28 @@
 authors = ["Kyle Simpson <kyleandrew.simpson@gmail.com>"]
 description = "An async Rust library for the Discord voice API."
 documentation = "https://docs.rs/songbird"
-edition = "2018"
+edition = "2021"
 homepage = "https://github.com/serenity-rs/songbird"
 include = ["src/**/*.rs", "Cargo.toml", "build.rs"]
 keywords = ["discord", "api", "rtp", "audio"]
 license = "ISC"
 name = "songbird"
 readme = "README.md"
 repository = "https://github.com/serenity-rs/songbird.git"
-version = "0.3.0"
+version = "0.2.2"
+rust-version = "1.61"
 
 [dependencies]
 derivative = "2"
+pin-project = "1"
 serde = { version = "1", features = ["derive"] }
 serde_json = "1"
 tracing = { version = "0.1", features = ["log"] }
 tracing-futures = "0.2"
-symphonia-core = "0.5"
+
+[dependencies.once_cell]
+version = "1"
+optional = true
 
 [dependencies.async-trait]
 optional = true
@@ -45,28 +50,50 @@ version = "5"
 [dependencies.discortp]
 features = ["discord-full"]
 optional = true
-version = "0.4"
+version = "0.5"
 
-# Temporary hack to pin MSRV.
 [dependencies.flume]
 optional = true
 version = "0.10"
 
 [dependencies.futures]
 version = "0.3"
 
-[dependencies.parking_lot]
+[dependencies.lazy_static]
 optional = true
-version = "0.12"
+version = "1"
 
-[dependencies.pin-project]
+[dependencies.parking_lot]
 optional = true
-version = "1"
+version = "0.12"
 
 [dependencies.rand]
 optional = true
 version = "0.8"
 
+[dependencies.reqwest]
+optional = true
+default-features = false
+features = ["stream"]
+version = "0.11"
+
+[dependencies.ringbuf]
+optional = true
+version = "0.2"
+
+[dependencies.rubato]
+optional = true
+version = "0.12"
+
+[dependencies.rusty_pool]
+optional = true
+version = "0.7"
+
+[dependencies.serde-aux]
+default-features = false
+optional = true
+version = "3"
+
 [dependencies.serenity]
 optional = true
 default-features = false
@@ -83,11 +110,29 @@ branch = "next"
 optional = true
 version = "1"
 
+[dependencies.symphonia]
+optional = true
+default-features = false
+version = "0.5"
+git = "https://github.com/FelixMcFelix/Symphonia"
+branch = "songbird-fixes"
+
+[dependencies.symphonia-core]
+optional = true
+version = "0.5"
+git = "https://github.com/FelixMcFelix/Symphonia"
+branch = "songbird-fixes"
+
 [dependencies.tokio]
 optional = true
 version = "1.0"
 default-features = false
 
+[dependencies.tokio-util]
+optional = true
+version = "0.7"
+features = ["io"]
+
 [dependencies.twilight-gateway]
 optional = true
 version = "0.12.0"
@@ -108,7 +153,7 @@ version = "2"
 
 [dependencies.uuid]
 optional = true
-version = "0.8"
+version = "1"
 features = ["v4"]
 
 [dependencies.xsalsa20poly1305]
@@ -118,7 +163,10 @@ features = ["std"]
 
 [dev-dependencies]
 criterion = "0.3"
+ntest = "0.8"
+symphonia = { version = "0.5", features = ["mp3"], git = "https://github.com/FelixMcFelix/Symphonia", branch = "songbird-fixes" }
 utils = { path = "utils" }
+tokio = { version = "1", features = ["rt", "rt-multi-thread"] }
 
 [features]
 # Core features
@@ -128,45 +176,48 @@ default = [
     "gateway",
 ]
 gateway = [
-    "gateway-core",
-    "tokio/sync",
-    "tokio/time",
-]
-gateway-core = [
     "dashmap",
     "flume",
+    "once_cell",
     "parking_lot",
-    "pin-project",
-]
-driver = [
-    "async-tungstenite",
-    "driver-core",
-    "tokio/fs",
-    "tokio/io-util",
-    "tokio/macros",
-    "tokio/net",
-    "tokio/process",
-    "tokio/rt",
     "tokio/sync",
     "tokio/time",
 ]
-driver-core = [
+driver = [
     "async-trait",
+    "async-tungstenite",
     "audiopus",
     "byteorder",
     "discortp",
+    "reqwest",
     "flume",
+    "lazy_static",
     "parking_lot",
     "rand",
+    "ringbuf",
+    "rubato",
+    "serde-aux",
     "serenity-voice-model",
     "streamcatcher",
+    "symphonia",
+    "symphonia-core",
+    "rusty_pool",
+    "tokio-util",
+    "tokio/fs",
+    "tokio/io-util",
+    "tokio/macros",
+    "tokio/net",
+    "tokio/process",
+    "tokio/rt",
+    "tokio/sync",
+    "tokio/time",
     "typemap_rev",
     "url",
     "uuid",
     "xsalsa20poly1305",
 ]
-rustls = ["async-tungstenite/tokio-rustls-webpki-roots", "rustls-marker"]
-native = ["async-tungstenite/tokio-native-tls", "native-marker"]
+rustls = ["async-tungstenite/tokio-rustls-webpki-roots", "reqwest/rustls-tls", "rustls-marker"]
+native = ["async-tungstenite/tokio-native-tls", "native-marker", "reqwest/native-tls"]
 serenity-rustls = ["serenity/rustls_backend", "rustls", "gateway", "serenity-deps"]
 serenity-native = ["serenity/native_tls_backend", "native", "gateway", "serenity-deps"]
 twilight-rustls = ["twilight", "twilight-gateway/rustls-native-roots", "rustls", "gateway"]
@@ -180,8 +231,6 @@ rustls-marker = []
 native-marker = []
 
 # Behaviour altering features.
-youtube-dlc = []
-yt-dlp = []
 builtin-queue = []
 
 # Used for docgen/testing/benchmarking.
@@ -191,6 +240,7 @@ internals = []
 [[bench]]
 name = "base-mixing"
 path = "benches/base-mixing.rs"
+required-features = ["internals"]
 harness = false
 
 [[bench]]

Makefile.toml

@@ -26,6 +26,10 @@ dependencies = ["format"]
 [tasks.build-variants]
 dependencies = ["build", "build-gateway", "build-driver"]
 
+[tasks.check]
+args = ["check", "--features", "full-doc"]
+dependencies = ["format"]
+
 [tasks.clippy]
 args = ["clippy", "--features", "full-doc", "--", "-D", "warnings"]
 dependencies = ["format"]