README.md

wasm-audio-worklet

Proof-of-concept (to-be-made-template) for creating Web Audio Worklets that run DSP from WASM generated by Rust. Rust -> WASM -> Audio Worklet -> threaded realtime audio in the browser

(with hopes that there may also be some performance gain in using WASM instead of JS AudioWorklets but we'll see?)

Directory Map

• ./:
  • the root of the project is an npm module that is intended to be a portable rust/wasm AudioWorkletNode
  • Intended to be a simple npm install for other JS projects to import this module so the wasm + worklet are bundled too
  • presently it exposes a new AudioWorkletNode
  • public/rust_wasm.wasm and public/worklet.js get bundled as well as they must be loaded statically
  • The AudioNode gets reference to their paths via URL(...).href
• ./rust-wasm - the rust crate we compile to wasm to perform DSP.
  • built via cargo build --target wasm32-unknown-unknown (notably not with wasm-pack)
• ./demo
  • a small vite vanilla-ts page for testing things

Things Tried + Resources

• Started here:
  • https://dev.to/speratus/i-built-this-despite-a-flaw-in-rusts-webassembly-toolchain-38p2
    • this repo is more akin to part 2 of this blog
    • I chose to avoid wasm-pack/wasm-bindgen (part 3) because:
      • it either involves mutating some of the JS bindings generated by wasm-pack (prone to breaking, this is not a public interface)
      • or poly-filling some things in the AudioWorklet environment/context so wasm-pack bindings don't error-out when init/initSync is called (I'm not sure what would be involved in polyfilling all this but it felt like potentially a lot or possibly infeasible. e.g. fetch API is not accessible to AudioWorklets)
    • This blog didn't get me the full way however: example here uses a wasm function call per-sample but I wanted/needed rust/wasm processing full audio blocks.
    • This requires some more convoluted memory management (so we can pass references to Float32Arrays between WASM and JS)
• This was also helpful: https://www.toptal.com/webassembly/webassembly-rust-tutorial-web-audio
  • However, this particular example doesn't produce audio from the wasm process fn (its doing pitch detection).
  • Wanted something that was mutating the output (so we can generate/modify the signal)
• This example from the wasm-bindgen book examples was helpful too: https://rustwasm.github.io/wasm-bindgen/examples/wasm-audio-worklet.html
  • but it depends on unstable cargo features
• Ultimately found: https://github.com/the-drunk-coder/wasm-loop-player/tree/master
  • which was exactly what I needed (and does more cool stuff - sample playback from wasm)
  • A more full example from the same person: https://github.com/the-drunk-coder/ruffbox
  • Both insipred by: this repo and this blog