Forced subtitle track may not be displayed and selectable by the user

[moneytoo]

ExoPlayer Version

2.18.0

Devices that reproduce the issue

• Samsung Galaxy S21 FE running Android 12

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Yes

Reproduction steps

1. Open the attached video file in the demo app

Expected result

The mkv file contains a single subtitle track with forced flag. I would expect the default behavior of StyledPlayerView that track would be either in full control of the user or at least that the track would be active/visible.

Actual result

Because the subtitle track has forced flag, the subtitle selector is disabled (#9432, since r2.18.0). But because the track lacks the default flag, it is not selected and therefore not visible to the user.
In versions prior to r2.18.0, user could just open the subtitle selector and enable this track.

I fully understand that ExoPlayer as well as the ui library is more tailored for streaming solutions where operators are in control of the media. So this is just my case for your consideration as files like these (in this case mkv bluray rips) exists and should be valid. It would be just as nice to be able to handle user media files in the StyledPlayerView.

Media

forced.zip

Bug Report

• You will email the zip file produced by adb bugreport to dev.exoplayer@gmail.com after filing this issue.

[icbaker]

Thanks for the detailed explanation and repro media!

Expected result

The mkv file contains a single subtitle track with forced flag. I would expect the default behavior of StyledPlayerView that track would be either in full control of the user or at least that the track would be active/visible.

Actual result

Because the subtitle track has forced flag, the subtitle selector is disabled (#9432, since r2.18.0). But because the track lacks the default flag, it is not selected and therefore not visible to the user.

ExoPlayer's DefaultTrackSelector should select a forced, non-default subtitle track if the subtitle track's language matches the selected audio track's language:

ExoPlayer/library/core/src/main/java/com/google/android/exoplayer2/trackselection/DefaultTrackSelector.java

Line 3550 in 03569f9

|| (isForced && selectedAudioLanguageScore > 0);

In the forced.mkv media provided, it looks like the text track declares its language as English but the audio track does not declare a language:

$ mediainfo forced.mkv

Audio
ID                                       : 2
Format                                   : DTS
Format/Info                              : Digital Theater Systems
Codec ID                                 : A_DTS
Duration                                 : 10 s 7 ms
Bit rate mode                            : Constant
Bit rate                                 : 1 509 kb/s
Channel(s)                               : 6 channels
Channel layout                           : C L R Ls Rs LFE
Sampling rate                            : 48.0 kHz
Frame rate                               : 93.750 FPS (512 SPF)
Bit depth                                : 24 bits
Compression mode                         : Lossy
Delay relative to video                  : -795 ms
Stream size                              : 1.15 GiB
Default                                  : Yes
Forced                                   : No

Text
ID                                       : 3
Format                                   : UTF-8
Codec ID                                 : S_TEXT/UTF8
Codec ID/Info                            : UTF-8 Plain Text
Duration                                 : 7 s 513 ms
Bit rate                                 : 2 b/s
Count of elements                        : 61
Stream size                              : 1.30 KiB (0%)
Title                                    : English Forced
Language                                 : English
Default                                  : No
Forced                                   : Yes

I think there's a couple of approaches here:

1. Leave StyledPlayer(Control)View behaviour as-is, and tweak DefaultTrackSelector to slightly relax the language check when evaluating a forced subtitle track for selection. I haven't thought too deeply about this but maybe:

   Select a forced subtitle track if either its language or the selected audio track language are unknown, and there's no subtitle track with a 'better matched' language.

   Or maybe it needs special-casing to when we only have a single audio track?

   This will implement this part of your 'expected result':

   at least that the track would be active/visible

2. Change StyledPlayer(Control)View as suggested to allow the user to toggle forced subtitle tracks whether or not they're selected by the player (or maybe only if the player has decided not to select them?).

I think I prefer (1) - it seems reasonable to me that the player should select the forced subtitle track in the media you've provided.

@tonihei Do you have any thoughts on my suggested changes to the DefaultTrackSelector logic?

[tonihei]

I think the proposed logic change makes sense. The HLS spec for example only says

When selecting a FORCED Rendition, a client SHOULD choose the one that best matches the current playback environment (e.g., language).

Having one of the languages undefined constitutes a "match with the playback environment" I'd say.

This means the evaluation for selectedAudioLanguageScore in TextTrackInfo needs to have decreasing scores for these cases:

• A match or partial match of the audio and text track languages (using the existing logic in getFormatLanguageScore)
• A match of two undefined languages. This is already implemented by setting allowUndeterminedFormatLanguage to true if selectedAudioLanguageUndetermined.
• A pair of a defined and an undefined language. This "match level" is new to handle the case here.
• A mismatch between two defined languages. This should get a score of 0 to disallow the selection of forced tracks even if they exist (as already implemented).

We could add this option with another parameter in getFormatLanguageScore for example?

In any case, if we make this change, the problem above should be fixed and it should also work for a mixture of defined and undefined languages because any defined match will always be preferred over a (partially) undefined one even if multiple forced texts are eligible for selection.