Sidecar load WebVTT

[sanbornhilland]

It would be useful to be able to sidecar load webvtt by providing an alternative url from which to fetch them in addition to getting them from the .mpd. Thoughts?

[dougdoe]

Note: We require this behaviour to be able to late-bind Closed Captions to a stream. This is a side effect of existing content we already have ingested. We receive the MPD and WebVTT URLs separately, so we would like to be able to attach a WebVTT URL to the content.

[tdrews]

It seems reasonable to have a separate subtitles URL. I think the cleanest implementation would be to add a subtitles URL to the StreamVideoSource constructor, which would of course require knowing the subtitles URL before playback.

@dougdoe: Although you are fetching the MPD and subtitles separately do you have both before playback? Or do you want to be able to fetch subtitles during playback?

[dougdoe]

We will have both the MPD and subtitle URL(s) prior to playback, so for our purposes your suggestion would be appropriate. In terms of priority, this is high priority for us – if needed, perhaps @sanbornhnewyyz can work with you on it.

Thanks!

[tdrews]

I think we can do this for v1.5.0, which we have slotted for mid August.

[dougdoe]

Sounds good. We can give it a try when we see the commit go through. Thanks.

[sanbornhilland]

This works great but only for one track as far as I can tell. If I call it multiple times then only the last one will get used. I wouldn't be able to add say, a french and an english track. Is that correct? If so, would it be possible to make it so that we can call this multiple times or have it accept an array of vtt urls?

[dougdoe]

We have also determined that the WebVTT URLs that we are receiving are in the form of a playlist, which contains multiple .vtt files. Thus, we will have one playlist for each language (e.g. one playlist for English, one for Spanish, one for French).

Apologies for these lack of details. I can send you an example playlist via email, if it helps. Let me know.

[tdrews]

With the current API you could parse the playlists in the application and call addExternalCaptions() with each subtitles URL. I'd like to avoid implementing anything too specific. Are the playlists in a standard file format?

[dougdoe]

Yes, I believe the playlists are standard format. An example is below. Each playlist contains many .vtt files for a single language.

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:9
#EXTINF:9.0,
sub000000.vtt
#EXTINF:7.2,
sub000001.vtt
#EXTINF:9.0,
sub000002.vtt
#EXTINF:7.2,
sub000003.vtt
#EXTINF:9.0,
sub000004.vtt
#EXTINF:7.2,
sub000005.vtt
#EXTINF:9.0,
sub000006.vtt
#EXTINF:7.2,
sub000007.vtt
#EXTINF:9.0,
sub000008.vtt
#EXTINF:8.8,
sub000009.vtt
#EXTINF:9.0,
sub000010.vtt
#EXTINF:8.0,
sub000011.vtt
#EXTINF:9.0,
sub000012.vtt
#EXTINF:7.2,
sub000013.vtt
#EXTINF:9.0,
sub000014.vtt
#EXTINF:7.2,
sub000015.vtt
#EXTINF:9.0,
sub000016.vtt
#EXTINF:7.2,
sub000017.vtt
#EXTINF:9.0,
sub000018.vtt
#EXTINF:8.9,
sub000019.vtt
#EXTINF:9.0,
sub000020.vtt
#EXTINF:7.2,
sub000021.vtt
#EXTINF:9.0,
sub000022.vtt
#EXTINF:8.8,
sub000023.vtt
#EXTINF:9.0,
sub000024.vtt
#EXTINF:7.2,
sub000025.vtt
#EXTINF:9.0,
sub000026.vtt
#EXTINF:7.2,
sub000027.vtt
#EXTINF:9.0,
sub000028.vtt
#EXTINF:7.2,
sub000029.vtt
#EXTINF:9.0,
sub000030.vtt
#EXTINF:7.2,
sub000031.vtt
#EXTINF:9.0,
sub000032.vtt
#EXTINF:7.2,
sub000033.vtt
#EXTINF:9.0,
sub000034.vtt
#EXTINF:7.2,
sub000035.vtt
#EXTINF:9.0,
sub000036.vtt
#EXTINF:7.5,
sub000037.vtt
#EXTINF:9.0,
sub000038.vtt
#EXTINF:7.2,
sub000039.vtt
#EXTINF:9.0,
sub000040.vtt
#EXTINF:7.2,
sub000041.vtt
#EXTINF:9.0,
sub000042.vtt
#EXTINF:7.2,
sub000043.vtt
#EXTINF:9.0,
sub000044.vtt
#EXTINF:6.3,
sub000045.vtt
#EXT-X-ENDLIST

[tdrews]

I see. I had originally thought that each playlist pointed to a number of completely separate subtitles files. However, it looks like the playlist points to one subtitles track that is separated into multiple files. I believe the work required for supporting this scheme (correctly) overlaps with the work required to support TTML (#111); so it's a little beyond what we've scoped out for v1.5.0.

I will add an issue to track support for M3U based subtitles. In the meantime I think support for this scheme can be accomplished at the application level, e.g., by stitching each vtt file together as a string and passing a data URI to addExternalCaptions().

[dougdoe]

Thanks, this seems like a reasonable short-term strategy. I think the only outstanding question for this issue now is from @sanbornhnewyyz above. He indicates:

If I call addExternalCaptions() multiple times then only the last one will get used.
I wouldn't be able to add say, a french and an english track. Is that correct?

Is this right, or is he not using the API correctly?

[tdrews]

@sanbornhnewyyz was right. However, we've updated the API in 950b173.