Expose AudioContext in Worker/ServiceWorker

[guest271314]

Describe the feature
Expose AudioContext in Worker and ServiceWorker contexts.

Is there a prototype?
No.

Currently users need to use Native Messaging to run local media players, for example mpv https://github.com/mpv-player/mpv; see https://github.com/mpv-player/mpv/blob/bc9d556f3a890cf5f99e9dced0117e2d8a91ff09/DOCS/man/javascript.rst, https://github.com/Kagami/mpv.js.

Describe the feature in more detail
The ability to use AudioContext in a Worker, particularly in a ServiceWorker context.

See https://bugs.chromium.org/p/chromium/issues/detail?id=1131236.

Use cases:

• Developers would like extensions built with service workers to continue to support the ability to play and control audio playback initiated from a background context.
• It is a podcast aggregator and it plays audio from the background page so that the user can close its main page and continue to listen to the podcast.
• Para mi es una excelente opción el tener un podcast de fondo mientras trabajo, de la misma forma que uso una extensión para indicarme cuando me llegan nuevos mensajes mi correo o ver mi gmail en una mini ventana que se activa en el icono de la extensión.
• In general, playing audio without creating an different PictureInPictuireWindow, Window, <iframe> or a dedicated Document just to play audio.

[padenot]

AudioWG calls:

The worker case is WebAudio/web-audio-api-v2#16, and this is agreed upon and will happen and yes, transferable MediaStream (not just ReadableStream/WriteableStreams) will happen.

For the service worker case, it's really unclear. If an app starts playing audio, and then all tabs for this app are closed, how do you pause the audio? There is no way to display a UI element.

[rtoy]

Teleconf: As we already agreed on supporting a Worker, we'll work on that first, and ServiceWorker at a lower priority while we gather info on use cases and implications of supporting that.

[JohnWeisz]

I'd like to add in another use case here: performance in complex production apps.

If AudioContext is available in worker threads, it means that a worker thread can be dedicated specifically to schedule events on the AudioContext (param automation, creating/connecting nodes), without any interference caused by potential main thread load, most commonly caused by UI/DOM (which can be optimized by itself of course).

[GeorgeTailor]

how is this different from #2423 ?

[GeorgeTailor]

As per my understanding WorkerGlobalScope is an abstraction over both Web Worker and Service Worker. #2423 requests BaseAudioContext to be exposed to WorkerGlobalScope.

[hoch]

2022 TPAC:
The Worker support for BaseAudioContext is already planned. We'll keep this issue for the future discussion for the ServiceWorker support.

[padenot]

2023 TPAC Audio WG Discussion:

The WG will not pursue this. ServiceWorker has nothing to do with audio playback and processing. The Web Worker support is planned (#2423).

[hoch]

To add:

To make this idea work, the lifetime of ServiceWorker and how it interacts with BaseAudioContext needs to be clearly defined. The WG believes that this line of work is outside the scope of the current charter.

[voxpelli]

2023 TPAC Audio WG Discussion:

The WG will not pursue this. ServiceWorker has nothing to do with audio playback and processing. The Web Worker support is planned (#2423).

@padenot @hoch It was left open before to track this request, any suggestion of where else a request for some kind of audio support in the background can be filed? Through ServiceWorker, SharedWorker or something else.

[hoch]

@voxpelli The roadmap for this work is unclear to me because of the complexity of the interaction between ServiceWorker and AudioContext. I think we need at least two things to reopen and reprioritize this issue in the Audio WG:

• Is the working group responsible for ServiceWorker interested in this work? This line of work will produce significant change/addition to both specs.
• Do we have a strong demand from developers who have real-world use cases?

Also, closing this issue only means that the WG has other priorities. When the priority changes in the future, the group will definitely reopen this and invite more opinions.

@padenot Please feel free to add any other rationales you have in mind.

[padenot]

I generally agree with what @hoch said.

Audio playback on the web is fundamentally tied to an active document for a host of reasons, and this would be a fundamental change to how a User-Agent is expected to behave by its users.

To take a parallel, on mobile or desktop, in native, there's always something (an app, a command-line program, a widget, or sometimes the system itself in the case of a notification) that is the cause of the sound, and the user can understand what's going on and can easily interact with the thing that is making sound, for example to pause the audio playback.

I'm not saying it's impossible, but rather than there are significant challenges to overcome before this can be worked on.

It's also unclear to me what use-cases this would solve.

[padenot]

Extensions have documents and ways to interact with the code that runs via widgets. MediaStreamTrackGenerator and MediaStreamTrackProcessor are always tied to a document. They can be instantiated in a Worker, but a Worker lifetime is tied to the life-time of its document.

Media Session API requires a document and the lifetime is tied to this document. All of this is a lifetime problem.

If you want to be able to get a tab out of the way, ask the browser vendor to provide a way to do so. This has nothing to do with audio playback from a service worker.

[yume-chan]

Not being able to use AudioContext in ServiceWorker gives me the impression that the standards team doesn't want web apps to have the same capabilities as traditional Desktop apps.

For example, in a web-based music streaming app, I can use BroadcastChannel, SharedWorker or ServiceWorker to make multiple tabs to communicate and coordinate with each other, like displaying the same now-playing information and controlling the playback from any tabs.

But the playback itself must happen in one of the tabs, if the user closes that tab, the playback stops. I can't resume it in another tab because starting an AudioContext requires user activation (even if I can, this make the code more complex, and might have a short pause between switching). This creates an imperfect experience for users: "Why can I close all these tabs without affecting the playback, but I can't close that one? On desktop version of [insert music app here], I can close all its windows and the music will continue playing".

I'm not talking about "retain service worker and play audio from it even if it has 0 clients/documents", only allowing tabs to be freely closed/reloaded (because service worker can live through reloading last tab) would be a huge improvement.

---

Similar story for Media Session API. I have used https://github.com/sinono3/souvlaki to integrate OS media control in an Electron app, for all of Windows, Linux and macOS, they don't need a window to have media session displayed and controlled.