Add support for preferred video role flags

[soldierinwhite]

Use case description

Some manifests can include alternative video tracks for example for sign language interpretation of the same video. The manifest parser is able to parse the roles included on these representations and is accessible via format.roleFlags. A client should be able to give a preferred role flag for video from those defined in C.java to the track selector in the same way preferred language or audio role flags are catered for. Right now this is possible with setting a custom selection override, but that also entails including all other filters such as bitrate and viewport width filtering in a custom override rather than using the default trackselector's built-in support.

Proposed solution

DefaultTrackSelector.ParametersBuilder would have a setPreferredVideoRoleFlags(int flags) that would accept single or combined flags for instance setPreferredVideoRoleFlags(ROLE_FLAG_ALTERNATE|ROLE_FLAG_SIGN). In the case of a single flag the track selector would select tracks that may have more flags than those specified, but includes at least all of the specified flags, so in the example above, it would be able to select a track with flags=ROLE_FLAG_SIGN|ROLE_FLAG_ALTERNATE|ROLE_FLAG_SUPPLEMENTARY but not one with flags=ROLE_FLAG_ALTERNATE|ROLE_FLAG_EASY_TO_READ. If the specified flags are not available on any video track, revert to ROLE_FLAG_MAIN or 0, which is the default flag if none is specified.

Alternatives considered

The order of filtering related to other parameters, such as maxVideoBitrate, might be a point that needs some thought. Should flags be filtered on after bitrate filters are already applied or rather bitrate filter be applied after role flags are already filtered. In my opinion the latter is preferable, as that would be most compatible with user needs.

[ojw28]

The track selector would select tracks that may have more flags than those specified, but include at least all of the specified flags

Would you expect this logic for audio and text as well, because we don't enforce the part in bold for those at the moment. We should probably make sure all three work in the same way, else it's going to be pretty confusing for someone who's using more than one of them.

[soldierinwhite]

Would you expect this logic for audio and text as well, because we don't enforce the part in bold for those at the moment. We should probably make sure all three work in the same way, else it's going to be pretty confusing for someone who's using more than one of them.

I didn't know that, but in my opinion that would be best. Since exoplayer parses different tags in DASH adaptation sets to one scheme of flags, you will often see adaptation sets with an accessibility tag that gets one flag, and a role tag that gets another flag. An exoplayer client might just be interested in the accessibility for instance, say ROLE_FLAG_SIGN, so then a role flag like ROLE_FLAG_ALTERNATE shouldn't have to be specified as well, since the client would be equally interested in having that track selected if it had ROLE_FLAG_MAIN or no role tag whatsoever. Imagine a service where you have dual medium content (default video translated to sign language), but also content specifically made for accessibility (only sign language exists). One user setting to choose sign language should easily be able to cover both cases.

If the preferred flags had to be a perfect match, it would mean that such clients where the flags can come in a variety of combinations would still have to implement custom selection overrides that implements this type of behavior.

Edit: If your meaning was that any of the preferred tags added would be filtered on, that might be less problematic. I think it would be acceptable that that would be the case if that is the case for audio right now.

[ojw28]

To clarify, our current logic for audio and text does not require perfect matching. It does a bit-wise AND between the requested flags and the flags defined by the media track, and counts the number of set bits in the result. A track with a higher count is preferred to a track with a lower one. Suppose the requested flags are X and Y:

• If there are tracks that declare X and Y, they will be preferred over other tracks.
• There is no penalty for tracks that also declare additional flags. A track that declares X, Y and Z will be scored the same as one that declares X and Y exactly.
• A track that only declares one of X and Y will be preferred over a track that declares neither of them.

The third bullet point was the distinction I was drawing between our current logic and your description. In your description, you stated that a track should "include at least all of the specified flags" to be chosen, where-as we'll prefer to choose a partial match over a track that declares none of the flags.

[soldierinwhite]

Thanks for the clarification. In that case I prefer the current logic, you should keep the same for video as well. That is much better than what I envisaged. I guess the only use case this does not cater for is when a specific track I am looking for is identified by more than one flag, but I am not interested in other tracks that only have some of the flags. But that seems like an edge case that is hard to cater for.

[ojw28]

Thanks! There is one small change I'd propose for the existing logic, which is to boost the "exact match" case so that tracks whose flags match exactly are preferred over tracks that also declare additional flags, in the case that both are present.

[soldierinwhite]

That sounds like a great improvement.

[tonihei]

Added support for preferred video role flags now and update existing role flag logic as described above (=prefer perfect matches over partial matches). Note that preferred video flags are only applied for fixed video track selections and will not be used for adaptive playbacks. This is due to the same problem as #9519 and we will use #9519 to track fixing this for both cases.