Experimental support for the video-layers-allocation extension #3504
+215
−35
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds experimental support for the
rtp-hdrext/video-layers-allocation00RTP extension:https://webrtc.googlesource.com/src/+/refs/heads/main/docs/native-code/rtp-hdrext/video-layers-allocation00/
As explained in the documentation above, this extension, when negotiated, provides info associated with a simulcast/SVC topology, including the number of available spatial and temporal layers, info on the target bitrates, and the associated resolutions. To keep things simple, when we negotiate this extension we only extract the number of spatial and temporal layers, and then make it available to plugins as part of the
extensionsstructure.For a POC integration, I added support to this extension to the EchoTest plugin, which always tries to negotiate it. When it finds VLA info associated to an incoming packet, it enriches the existing simulcast/SVC events with that info too (
total_spatial_layers,total_temporal_layers), that the EchoTest demo then uses to dynamically show/hide the associated buttons in the demo page.Should this prove to work nicely, we can add the same functionality to the VideoRoom plugin as well, where I can definitely see this information being useful to interested subscribers (dynamically or as part of the info the plugin advertises about publishers). In the future, we can consider parsing and advertising bitrates/resolutions too, but that would be considerably more work: just notifying about existing layers is already a useful piece of information with very little overhead.