WebTransport API and ManagedMediaSource API

[aboba]

The proposed ManagedMediaSource API includes eventHandlers for onqualitychanged, onstartstreaming and onendstreaming.

How would a WebTransport-based client (e.g. MoQ) respond to these events?

WWDC Presentation is here.

[jan-ivar]

What is your question for this WG? This seems like MSE, whereas WebTransport is a byte transport with no concept of media.

[wilaw]

@aboba - how would the server know what the client's burst window is i.e the time between the onstartstreaming and onendstreaming events?? Do you anticipate the client listening to these events and then communicating them to the server via some type of control channel?

[aboba]

@wilaw My concern is whether WebTransport provides acceptable power consumption and thermals on mobile devices.
This is important because some key WebTransport/MoQ scenarios (e.g. sports, cloud games) are likely to be watched on mobile devices. In situations where CMAF is transported over WebTransport, the API used to decode and render was typically MSE.

However, power consumption and thermals turned out to be an issue with the original MSE API, which has now been enhanced with ManagedMediaSource API, designed to manage power consumption of streaming implementations on mobile devices. Managed Media Source works by providing events to applications that enable them to bundle their reads during periods when the cellular modem is awake. As a result, when WebTransport API becomes available on mobile browsers, mobile WebTransport applications that decode and render CMAF (e.g. MoQ) will also be likely to decode and render via ManagedMediaSource API.

However, it isn't clear to me how to "bundle" WebTransport traffic in the way that MMS API envisages. For example, the startstreaming event indicates that more media data should now be loaded from the network. But it's not clear to me that waiting to call reader.read() within the onStartStreamingHandler below is going to reduce power/thermals or just result in queue buildup in the network leading to loss and delay:

mediaSource.addEventListener(“startstreaming”, onStartStreamingHandler);

Similarly, the endstreaming event is the counterpart of startstreaming and signals that for now no more media data should be requested from the network. But what should a WebTransport application do in response to this event?

mediaSource.addEventListener(“endstreaming”, onStopStreamingHandler);

Overall, MMS seems oriented toward HLS, where the application requests content from the server. In WebTransport, the server will send to client without knowledge of when the client's modem is awake, and it is not clear to me how to coordinate the server and client.

Maybe I'm missing something, but I don't see a good way for WebTransport and MoQ to manage power consumption and thermals on mobile devices using ManagedMediaSource API. Maybe someone in the WG (e.g. from Apple?) can enlighten me.

[aboba]

@jyavenard @marcoscaceres Any thoughts on this?

[jyavenard]

There's no requirement to strictly follow the onstartstreaming onendstreaming event. They are hints.

on iOS devices what it means is that the 5G modem won't be used and instead it uses 4G as it has a lower power usage.

So if you ignore it, it will work just the same from a user perspective.

Just like streaming a live stream with MMS, there's no expectations that you would only fetched content between the two events.

As you mentioned, the primary aim was to allow sites to switch from a HLS stream to a MMS one with no power regression

[jan-ivar]

Meeting:

• You have less control of when you wake up in WebTransport. Except maybe reject new incoming streams.
• Apps in general can switch from server push to client pull to gain control of timing
• @wilaw to follow up

[wilaw]

@jyavenard - can you clarify on an iOS device whether the 5G modem access granted to HTTP FETCH requests between the hinted events, will also apply to data transmitted over WebTransport, both sending from client to server and receiving from server to client?

[jyavenard]

@jyavenard - can you clarify on an iOS device whether the 5G modem access granted to HTTP FETCH requests between the hinted events, will also apply to data transmitted over WebTransport, both sending from client to server and receiving from server to client?

There's no fine grain control such as one packet goes over 5g while another would simultaneously go on 4g. It's on or the other.

While a ManagedMediaSource is active, if you fetch content between the two events: they are flagged as media, and the system may turn on 5G.
So if the fetch has access to 5G, any other traffic transiting at the same time will also go over 5G

[wilaw]

@aboba - Jean-Yves's reply indicates that traffic over WebTransport would follow traffic via Fetch during the window between onstartstreaming and onendstreaming.. An application could monitor these MMS events and adjust its WebTransport send behavior appropriately. Are you OK with closing this issue?

[aboba]

@wilaw It's ok to close the "question".

Based on the MoQ WG discussion at IETF 119 and Jean-Yves' reply, it seems possible to write a sample combining WebTransport w/Fetch-like client requests and Managed MSE, utilizing the onstartstreaming and onendstreaming events. All the pieces are not yet in place in Safari TP, but they are in progress.

Unless Managed MSE has some implicit dependency to HLS, it shouldn't make a make a difference whether the WebTransport is left open.