Add support for extracting subtitles from MPD manifests #24517

Lukas0907 · 2020-03-28T18:21:16Z

Before submitting a pull request make sure you have:

At least skimmed through adding new extractor tutorial and youtube-dl coding conventions sections
Searched the bugtracker for similar pull requests
Checked the code with flake8

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

I am the original author of this code and I am willing to release it under Unlicense
I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

Bug fix
Improvement
New extractor
New feature

Description of your pull request and other information

This PR adds support for extracting subtitles from MPD manifests. The added code is similar to to _parse_mpd_formats() but only handles subtitles.

dstftw · 2020-03-31T07:31:17Z

No code copypasting.
Both must be extracted via the same routine with a single pass.

Lukas0907 · 2020-04-01T17:00:47Z

Thanks for the review. I will fix it.

Lukas0907 · 2020-04-02T12:32:35Z

@dstftw I have refactored the code, wrote a test and force pushed. Please take a look.

Lukas0907 · 2020-05-08T08:08:03Z

@dstftw Hi, any chance to get this merged? Thanks!

dstftw · 2020-05-08T08:12:50Z

youtube_dl/extractor/common.py

+    def _parse_mpd_formats(self, *args, **kwargs):
+        return self._parse_mpd_formats_subtitles(*args, **kwargs)[0]
+
+    def _parse_mpd_formats_subtitles(self, mpd_doc, mpd_id=None, mpd_base_url='', formats_dict={}, mpd_url=None):


Must return info dict.

dstftw · 2020-05-08T08:13:15Z

youtube_dl/extractor/common.py

            mpd_doc, mpd_id=mpd_id, mpd_base_url=mpd_base_url,
            formats_dict=formats_dict, mpd_url=mpd_url)

-    def _parse_mpd_formats(self, mpd_doc, mpd_id=None, mpd_base_url='', formats_dict={}, mpd_url=None):
+    def _parse_mpd_formats(self, *args, **kwargs):
+        return self._parse_mpd_formats_subtitles(*args, **kwargs)[0]


dstftw · 2020-05-08T08:18:09Z

test/test_InfoExtractor.py

+                'subtitles',
+                'https://example.com/streams/1/playlist/playlist.mpd',  # mpd_url
+                'https://example.com/streams/1/playlist',  # mpd_base_url
+                [{'acodec': 'mp4a.40.2',
+                  'asr': 48000,
+                  'container': 'm4a_dash',
+                  'ext': 'm4a',
+                  'filesize': None,
+                  'format_id': '131kbps',
+                  'format_note': 'DASH audio',
+                  'fps': None,
+                  'fragment_base_url': 'https://example.com/streams/1/playlist/',
+                  'fragments': [{'path': '../audio/1_stereo_131072/dash/init.mp4'},
+                                {'duration': 3989.0,
+                                 'path': '../audio/1_stereo_131072/dash/segment_0.m4s'},
+                                {'duration': 3989.0,
+                                 'path': '../audio/1_stereo_131072/dash/segment_1.m4s'}],
+                  'height': None,
+                  'language': 'de',
+                  'manifest_url': 'https://example.com/streams/1/playlist/playlist.mpd',
+                  'protocol': 'http_dash_segments',
+                  'tbr': 131.072,
+                  'url': 'https://example.com/streams/1/playlist/playlist.mpd',
+                  'vcodec': 'none',
+                  'width': None},
+                 {'acodec': 'mp4a.40.2',
+                  'asr': 48000,
+                  'container': 'm4a_dash',
+                  'ext': 'm4a',
+                  'filesize': None,
+                  'format_id': '196kbps',
+                  'format_note': 'DASH audio',
+                  'fps': None,
+                  'fragment_base_url': 'https://example.com/streams/1/playlist/',
+                  'fragments': [{'path': '../audio/1_stereo_196608/dash/init.mp4'},
+                                {'duration': 3989.0,
+                                 'path': '../audio/1_stereo_196608/dash/segment_0.m4s'},
+                                {'duration': 3989.0,
+                                 'path': '../audio/1_stereo_196608/dash/segment_1.m4s'}],
+                  'height': None,
+                  'language': 'de',
+                  'manifest_url': 'https://example.com/streams/1/playlist/playlist.mpd',
+                  'protocol': 'http_dash_segments',
+                  'tbr': 196.608,
+                  'url': 'https://example.com/streams/1/playlist/playlist.mpd',
+                  'vcodec': 'none',
+                  'width': None},
+                 {'acodec': 'none',
+                  'asr': None,
+                  'container': 'mp4_dash',
+                  'ext': 'mp4',
+                  'filesize': None,
+                  'format_id': '720p 1712kbps',
+                  'format_note': 'DASH video',
+                  'fps': 25,
+                  'fragment_base_url': 'https://example.com/streams/1/playlist/',
+                  'fragments': [{'path': '../video/720_1712128/dash/init.mp4'},
+                                {'duration': 4000.0,
+                                 'path': '../video/720_1712128/dash/segment_0.m4s'},
+                                {'duration': 4000.0,
+                                 'path': '../video/720_1712128/dash/segment_1.m4s'}],
+                  'height': 720,
+                  'language': None,
+                  'manifest_url': 'https://example.com/streams/1/playlist/playlist.mpd',
+                  'protocol': 'http_dash_segments',
+                  'tbr': 1712.128,
+                  'url': 'https://example.com/streams/1/playlist/playlist.mpd',
+                  'vcodec': 'avc1.42c00d',
+                  'width': 1280},
+                 {'acodec': 'none',
+                  'asr': None,
+                  'container': 'mp4_dash',
+                  'ext': 'mp4',
+                  'filesize': None,
+                  'format_id': '1080p 4669kbps',
+                  'format_note': 'DASH video',
+                  'fps': 25,
+                  'fragment_base_url': 'https://example.com/streams/1/playlist/',
+                  'fragments': [{'path': '../video/1080_4669440/dash/init.mp4'},
+                                {'duration': 4000.0,
+                                 'path': '../video/1080_4669440/dash/segment_0.m4s'},
+                                {'duration': 4000.0,
+                                 'path': '../video/1080_4669440/dash/segment_1.m4s'}],
+                  'height': 1080,
+                  'language': None,
+                  'manifest_url': 'https://example.com/streams/1/playlist/playlist.mpd',
+                  'protocol': 'http_dash_segments',
+                  'tbr': 4669.44,
+                  'url': 'https://example.com/streams/1/playlist/playlist.mpd',
+                  'vcodec': 'avc1.42c00d',
+                  'width': 1920}],
+                {'en': [{'ext': 'vtt',
+                         'url': 'https://example.com/streams/1/subtitles/sub_en.vtt'}],
+                 'fr': [{'ext': 'vtt',
+                         'url': 'https://example.com/streams/1/subtitles/sub_fr.vtt'}]},
            )


Code style.

Lukas0907 · 2020-05-11T14:49:43Z

@dstftw I tried to incorporate your proposed changes in 2508a4b, please have a look.

Authored by fstirlitz Modified from: ytdl-org/youtube-dl#6144 Closes: #73 Fixes: ytdl-org/youtube-dl#6106 ytdl-org/youtube-dl#14977 ytdl-org/youtube-dl#21438 ytdl-org/youtube-dl#23609 ytdl-org/youtube-dl#28132 Might also fix (untested): ytdl-org/youtube-dl#15424 ytdl-org/youtube-dl#18267 ytdl-org/youtube-dl#23899 ytdl-org/youtube-dl#24375 ytdl-org/youtube-dl#24595 ytdl-org/youtube-dl#27899 Related: ytdl-org/youtube-dl#22379 ytdl-org/youtube-dl#24517 ytdl-org/youtube-dl#24886 ytdl-org/youtube-dl#27215 Notes: * The functions `extractor.common._extract_..._formats` are still kept for compatibility * Only some extractors have currently been moved to using `_extract_..._formats_and_subtitles` * Direct subtitle manifests (without a master) are not supported and are wrongly identified as containing video formats * AES support is untested * The fragmented TTML subtitles extracted from DASH/ISM are valid, but are unsupported by `ffmpeg` and most video players * Their XML fragments can be dumped using `ffmpeg -i in.mp4 -f data -map 0 -c copy out.ttml`. Once the unnecessary headers are stripped out of this, it becomes a valid self-contained ttml file * The ttml subs downloaded from DASH manifests can also be directly opened with <https://github.com/SubtitleEdit> * Fragmented WebVTT files extracted from DASH/ISM are also unsupported by most tools * Unlike the ttml files, the XML fragments of these cannot be dumped using `ffmpeg` * The webtt subs extracted from DASH can be parsed by <https://github.com/gpac/gpac> * But validity of the those extracted from ISM are untested

dstftw added do-not-merge pending-fixes labels Mar 31, 2020

Add support for extracting subtitles from MPD manifests

fbfd54d

dstftw requested changes May 8, 2020

View reviewed changes

Refactor according to requests

2508a4b

dstftw force-pushed the master branch from 7b956a1 to 5e26784 Compare September 13, 2020 13:49

fstirlitz mentioned this pull request Apr 18, 2021

Subtitle extraction from streaming media manifests (HLS, DASH, Smooth Streaming) yt-dlp/yt-dlp#247

Merged

14 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for extracting subtitles from MPD manifests #24517

Add support for extracting subtitles from MPD manifests #24517

Lukas0907 commented Mar 28, 2020

dstftw commented Mar 31, 2020

Lukas0907 commented Apr 1, 2020

Lukas0907 commented Apr 2, 2020

Lukas0907 commented May 8, 2020

dstftw May 8, 2020

dstftw May 8, 2020

dstftw May 8, 2020

Lukas0907 commented May 11, 2020

Add support for extracting subtitles from MPD manifests #24517

Are you sure you want to change the base?

Add support for extracting subtitles from MPD manifests #24517

Conversation

Lukas0907 commented Mar 28, 2020

Before submitting a pull request make sure you have:

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

What is the purpose of your pull request?

Description of your pull request and other information

dstftw commented Mar 31, 2020

Lukas0907 commented Apr 1, 2020

Lukas0907 commented Apr 2, 2020

Lukas0907 commented May 8, 2020

dstftw May 8, 2020

Choose a reason for hiding this comment

dstftw May 8, 2020

Choose a reason for hiding this comment

dstftw May 8, 2020

Choose a reason for hiding this comment

Lukas0907 commented May 11, 2020