Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SVC related tweaks #3174

Merged
merged 5 commits into from
Mar 6, 2023
Merged

SVC related tweaks #3174

merged 5 commits into from
Mar 6, 2023

Conversation

lminiero
Copy link
Member

@lminiero lminiero commented Mar 1, 2023

This PR makes quite a few SVC related changes, since SVC was so far only limited to a custom VideoRoom demo that needed a specific room configuration too. Following the discussion in #3169, I decided to turn SVC more into a "first class citizen" instead, in order to allow it to be used pretty much as we do with simulcast today.

The first thing this PR does is add new helper functions for SVC processing, which work pretty much like the simulcast ones do: this way, we can use them independently in differenty plugins, rather than have a custom hardcoded behaviour as we did before. At the moment, it assumes VP9-SVC (since AV1 SVC is pretty much unusable right now), but we could extend it later on. These helper functions have then been integrated in both the EchoTest and VideoRoom plugins.

Another thing the PR does, now, is assume that the jsep property you send to a plugin can be extended with an array called svc, formatted like this:

{
    "type": "offer",    // same as before
    "sdp": "v=0...",    // same as before
    "svc": [
        { "mindex": 1, "mid": "1", "svc": "L3T3" }
    ]
}

This is needed because when you use SVC, the SDP doesn't reflect that, so if we want Janus (and the plugins) to know you're sending SVC, we need to say it out of band. Using custom messages in plugins was a possibility, but that would have forced each plugin to implement its own way, which I didn't like: better to have a core way to signal this, especially considering the jsep object does get to plugins, and so they can all process it the same way. This new svc array is quite basic, since it can contain a list of objects that say which stream being published is SVC, and treat it accordingly: at the time of writing, if you use janus.js, this property is set automatically (we iterate on transceivers before sending the SDP to Janus), but if you use your own stack or SDK you'll have to add it manually yourself instead.

To test all this in the EchoTest, just add a query string argument like ?vcodec=vp9&svc=L3T3: this will tell janus.js to use SVC for the video stream, enrich the jsep object accordingly, and allow you to change spatial/temporal layers exactly as the simulcast flavour of the demo currently does. Notice that this only works if you use the "regular" EchoTest demo: the Lua and Duktape versions don't support SVC yet.

For what concerns the VideoRoom, I got rid of the video_svc property you can set in rooms, which means now regular, simulcast and SVC streams can all coexist in the same room, without any prerequisite other than "the related codec must be supported in the room". As a consequence, I also got rid of the vp9svctest.html/.js demo, since it's useless now: you can publish SVC in the regular VideoRoom demos instead, using the same query string argument shown for the EchoTest (make sure vp9 is a supported video codec in your room, of course).

An example of simulcast and SVC publishers coexisting in the same room can be seen in the screenshot below:

Screenshot_2023-03-01_18-08-46

As you can see, both the publishers we supported too gave us options (we have buttons to select things), but one is using simulcast, and the other is using SVC, as the labels next to their current video resolution tells us.

Please do test this, especially if you're interested in testing SVC support, as I'd like to merge soon. Notice that, due to the amount of changes, this will remain a multistream only PR, as I don't plan to backport the same changes to 0.x too.

@lminiero lminiero added the multistream Related to Janus 1.x label Mar 1, 2023
@lminiero
Copy link
Member Author

lminiero commented Mar 6, 2023

Merging.

@lminiero lminiero merged commit ca36ad9 into master Mar 6, 2023
@lminiero lminiero deleted the svc-tweaks branch March 6, 2023 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
multistream Related to Janus 1.x
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant