Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add subtitle->audio track linking #471

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

rcombs
Copy link
Contributor

@rcombs rcombs commented Mar 28, 2021

Useful for cases where the appropriate tracks are otherwise ambiguous; see added notes.md section for a few examples.

Also clarified that TrackOverlay refers to tracks by their UIDs rather than their numbers; as far as I can tell this is the intent, but I can't find anything existing confirming this either way (nor any implementation code), so I could be wrong, but it seems consistent with other elements.

Previously it was suggested that TrackLink should go in a dedicated container element with maxOccurs=1, but I don't see any particular technical reason for that, and TrackOverlay already allows multiple instances without a container, though I can add one if that's a policy for newly-added elements or something.

@@ -454,6 +454,12 @@ the overlay track **SHOULD** be used instead. The order of multiple TrackOverlay
If not found it **SHOULD** be the second, etc.</documentation>
<extension type="webmproject.org" webm="0"/>
</element>
<element name="TrackLink" path="\Segment\Tracks\TrackEntry\TrackLink" id="0x6FAC" type="uinteger">
<documentation lang="en" purpose="definition">Specify that subtitle track is suitable for use with the audio Track UID specified (in the u-integer).
Multiple subtitle tracks may link to the same audio track, and the same subtitle track may link to multiple audio tracks by providing multiple TrackLink elements.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should mention that the TrackEntry it belongs to **MUST** be a subtitle track, in a normative way.


Track links are only necessary when the set of tracks available creates ambiguity about which
subtitles are appropriate for which audio, and function as a hint to the player to improve
its automatic track selection behavior. In cases where there is no such ambiguity, there is
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a link to the (#default-track-selection) section here. And probably augment the "Subtitle selection" section on how to use the TrackLink to solve such ambiguity.

@robUx4 robUx4 added format addition matroska-v5 spec_codecs Codec Matroska spec document target labels Mar 28, 2021
@robUx4
Copy link
Contributor

robUx4 commented Mar 28, 2021

The TrackOverlay refers to the SilentTrack feature which itself has SilentTrackNumber. This is a track number.

Given there's probably 0 use anywhere in the wild of this feature I don't mind the change. We may also change the name to TrackUIDOverlay or OverlayTrackUID.

Now given all of these are unused we might as well mark them as deprecated so they are not used while not specified properly or are of any use to anyone. This feature is basically used to replace parts of a track by another track the container level. This can be useful in video to encode a whole movie normally and have some parts with translated text on the video encoded in different tracks. But then it says the overlay picks the first available track, so I don't know how this can be of any use.

Maybe silently replacing parts of a video that have been removed by another part without reencoding, maybe even using a different codec. In this scenario the multiple TrackOverlay and the order to pick them would correspond to different edits of the video file.
You should move the TrackOverlay change in a separate PR as it's not used at all by the TrackLink feature.

For the TrackLink part, I think all the values should be held in a master element. When translating the spec to code, if the generated code has hierarchy (ie not a flat structure) it's easier to look for the master element and then its children, rather than iterating through all the elements of the TrackEntry to find all the TrackLink values.

@@ -454,6 +454,12 @@ the overlay track **SHOULD** be used instead. The order of multiple TrackOverlay
If not found it **SHOULD** be the second, etc.</documentation>
<extension type="webmproject.org" webm="0"/>
</element>
<element name="TrackLink" path="\Segment\Tracks\TrackEntry\TrackLink" id="0x6FAC" type="uinteger">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should have a minver="5" as it won't be in Matroska v4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
format addition matroska-v5 spec_codecs Codec Matroska spec document target
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants