[Vimeo] fix config parser regex #32742

gzz2000 · 2024-03-11T14:19:31Z

Please follow the guide below

You will be asked some questions, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your pull request (like that [x])
Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

Searched the bugtracker for similar pull requests
N/A Read adding new extractor tutorial
Read youtube-dl coding conventions and adjusted the code to meet them
Covered the code with tests (note that PRs without tests will be REJECTED)
Checked the code with flake8

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

I am the original author of this code and I am willing to release it under Unlicense
I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

Bug fix
Improvement
New extractor
New feature

Description of your pull request and other information

Vimeo's playerConfig now no longer ends in ; or \n, but can end directly with </script>.
A sample video for your testing: https://player.vimeo.com/video/792852679

<script>window.playerConfig = {"cdn_url": .................... }</script>

This fails the original regex (?s)\b(?:playerC|c)onfig\s*=\s*({.+?})\s*[;\n].
I modified it to (?s)\b(?:playerC|c)onfig\s*=\s*({.+?})\s*(?:;|\n|\<\/script\>) so it now works.

dirkf

Thanks for the suggestion.

I've proposed what I hope you'll agree is a better long-term solution.

The test video that you've mentioned should be added to the tests.

Output with alternative patch

$ python -m youtube_dl -vF 'https://player.vimeo.com/video/792852679'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-vF', u'https://player.vimeo.com/video/792852679']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: 8cd66b76f
[debug] Python 2.7.15 (CPython i686 32bit) - Linux-6.1.0-18-686-pae-i686-with-debian-12.5 - OpenSSL 1.1.1a  20 Nov 2018 - glibc 2.1.3
[debug] exe versions: ffmpeg 5.1.4-0, ffprobe 5.1.4-0
[debug] Proxy map: {}
[vimeo] 792852679: Downloading webpage
[vimeo] 792852679: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 792852679: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 792852679: Downloading fastly_skyfire m3u8 information
[vimeo] 792852679: Downloading fastly_skyfire m3u8 information
[vimeo] 792852679: Downloading akfire_interconnect_quic MPD information
[vimeo] 792852679: Downloading akfire_interconnect_quic MPD information
[vimeo] 792852679: Downloading fastly_skyfire MPD information
[vimeo] 792852679: Downloading fastly_skyfire MPD information
[info] Available formats for 792852679:
format code                                           extension  resolution note
hls-akfire_interconnect_quic_sep-audio-high-Original  mp4        audio only 
hls-fastly_skyfire_sep-audio-high-Original            mp4        audio only 
dash-akfire_interconnect_quic_sep-audio-a43f5b22      m4a        audio only DASH audio   66k , m4a_dash container, opus  (48000Hz)
dash-fastly_skyfire_sep-audio-a43f5b22                m4a        audio only DASH audio   66k , m4a_dash container, opus  (48000Hz)
dash-akfire_interconnect_quic_sep-audio-fdfeb1af      m4a        audio only DASH audio   98k , m4a_dash container, opus  (48000Hz)
dash-fastly_skyfire_sep-audio-fdfeb1af                m4a        audio only DASH audio   98k , m4a_dash container, opus  (48000Hz)
dash-akfire_interconnect_quic_sep-audio-68e873e0      m4a        audio only DASH audio  191k , m4a_dash container, mp4a.40.2 (48000Hz)
dash-fastly_skyfire_sep-audio-68e873e0                m4a        audio only DASH audio  191k , m4a_dash container, mp4a.40.2 (48000Hz)
hls-akfire_interconnect_quic-86                       mp4        426x240      86k , avc1.640015, 25.0fps, video only
hls-fastly_skyfire-86                                 mp4        426x240      86k , avc1.640015, 25.0fps, video only
dash-akfire_interconnect_quic-video-c7557046          mp4        426x240    DASH video  206k , mp4_dash container, avc1.640015, 25fps, video only
dash-akfire_interconnect_quic_sep-video-c7557046      mp4        426x240    DASH video  206k , mp4_dash container, avc1.640015, 25fps, video only
dash-fastly_skyfire-video-c7557046                    mp4        426x240    DASH video  206k , mp4_dash container, avc1.640015, 25fps, video only
dash-fastly_skyfire_sep-video-c7557046                mp4        426x240    DASH video  206k , mp4_dash container, avc1.640015, 25fps, video only
hls-akfire_interconnect_quic_sep-277                  mp4        426x240     277k , avc1.640015, 25.0fps, video only
hls-fastly_skyfire_sep-277                            mp4        426x240     277k , avc1.640015, 25.0fps, video only
hls-akfire_interconnect_quic-149                      mp4        640x360     149k , avc1.64001E, 25.0fps, video only
hls-fastly_skyfire-149                                mp4        640x360     149k , avc1.64001E, 25.0fps, video only
hls-akfire_interconnect_quic_sep-340                  mp4        640x360     340k , avc1.64001E, 25.0fps, video only
hls-fastly_skyfire_sep-340                            mp4        640x360     340k , avc1.64001E, 25.0fps, video only
dash-akfire_interconnect_quic-video-e08bead4          mp4        640x360    DASH video  378k , mp4_dash container, avc1.64001E, 25fps, video only
dash-akfire_interconnect_quic_sep-video-e08bead4      mp4        640x360    DASH video  378k , mp4_dash container, avc1.64001E, 25fps, video only
dash-fastly_skyfire-video-e08bead4                    mp4        640x360    DASH video  378k , mp4_dash container, avc1.64001E, 25fps, video only
dash-fastly_skyfire_sep-video-e08bead4                mp4        640x360    DASH video  378k , mp4_dash container, avc1.64001E, 25fps, video only
hls-akfire_interconnect_quic-259                      mp4        960x540     259k , avc1.64001F, 25.0fps, video only
hls-fastly_skyfire-259                                mp4        960x540     259k , avc1.64001F, 25.0fps, video only
hls-akfire_interconnect_quic_sep-450                  mp4        960x540     450k , avc1.64001F, 25.0fps, video only
hls-fastly_skyfire_sep-450                            mp4        960x540     450k , avc1.64001F, 25.0fps, video only
dash-akfire_interconnect_quic-video-83036911          mp4        960x540    DASH video  675k , mp4_dash container, avc1.64001F, 25fps, video only
dash-akfire_interconnect_quic_sep-video-83036911      mp4        960x540    DASH video  675k , mp4_dash container, avc1.64001F, 25fps, video only
dash-fastly_skyfire-video-83036911                    mp4        960x540    DASH video  675k , mp4_dash container, avc1.64001F, 25fps, video only
dash-fastly_skyfire_sep-video-83036911                mp4        960x540    DASH video  675k , mp4_dash container, avc1.64001F, 25fps, video only
hls-akfire_interconnect_quic-363                      mp4        1280x720    363k , avc1.640020, 25.0fps, video only
hls-fastly_skyfire-363                                mp4        1280x720    363k , avc1.640020, 25.0fps, video only
hls-akfire_interconnect_quic_sep-554                  mp4        1280x720    554k , avc1.640020, 25.0fps, video only
hls-fastly_skyfire_sep-554                            mp4        1280x720    554k , avc1.640020, 25.0fps, video only
dash-akfire_interconnect_quic-video-938c4a99          mp4        1280x720   DASH video 1009k , mp4_dash container, avc1.640020, 25fps, video only
dash-akfire_interconnect_quic_sep-video-938c4a99      mp4        1280x720   DASH video 1009k , mp4_dash container, avc1.640020, 25fps, video only
dash-fastly_skyfire-video-938c4a99                    mp4        1280x720   DASH video 1009k , mp4_dash container, avc1.640020, 25fps, video only
dash-fastly_skyfire_sep-video-938c4a99                mp4        1280x720   DASH video 1009k , mp4_dash container, avc1.640020, 25fps, video only
hls-akfire_interconnect_quic-535                      mp4        1920x1080   535k , avc1.64002A, 25.0fps, video only
hls-fastly_skyfire-535                                mp4        1920x1080   535k , avc1.64002A, 25.0fps, video only
hls-akfire_interconnect_quic_sep-726                  mp4        1920x1080   726k , avc1.64002A, 25.0fps, video only
hls-fastly_skyfire_sep-726                            mp4        1920x1080   726k , avc1.64002A, 25.0fps, video only
dash-akfire_interconnect_quic-video-bb63111a          mp4        1920x1080  DASH video 1960k , mp4_dash container, avc1.64002A, 25fps, video only
dash-akfire_interconnect_quic_sep-video-bb63111a      mp4        1920x1080  DASH video 1960k , mp4_dash container, avc1.64002A, 25fps, video only
dash-fastly_skyfire-video-bb63111a                    mp4        1920x1080  DASH video 1960k , mp4_dash container, avc1.64002A, 25fps, video only
dash-fastly_skyfire_sep-video-bb63111a                mp4        1920x1080  DASH video 1960k , mp4_dash container, avc1.64002A, 25fps, video only (best)
$

youtube_dl/extractor/vimeo.py

dirkf's suggestions

gzz2000 · 2024-03-12T01:39:53Z

Thanks for your quick response and valuable suggestion! The _search_json is great and I've made the requested changes.

As for the testing, there is already tests for such embedded videos and its json layout is now }}</script> as well. So I think there is no need to add another one.

dirkf · 2024-03-12T11:44:55Z

Thanks!

* https://github.com/ytdl-org/youtube-dl: [Vimeo] Improve `config` extraction (ytdl-org#32742)

fix vimeo config parser regex

3adfe21

dirkf requested changes Mar 11, 2024

View reviewed changes

youtube_dl/extractor/vimeo.py Outdated Show resolved Hide resolved

update vimeo.py for more robust json parsing, according to

7b8cb3e

dirkf's suggestions

gzz2000 requested a review from dirkf March 12, 2024 01:40

dirkf changed the title ~~fix vimeo config parser regex~~ [Vimeo] fix config parser regex Mar 12, 2024

dirkf merged commit a96a45b into ytdl-org:master Mar 12, 2024
14 checks passed

github-actions bot added a commit to hellopony/youtube-dl that referenced this pull request Mar 12, 2024

[bot] AutoMerging: merge all upstream's changes:

148e6a2

* https://github.com/ytdl-org/youtube-dl: [Vimeo] Improve `config` extraction (ytdl-org#32742)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Vimeo] fix config parser regex #32742

[Vimeo] fix config parser regex #32742

gzz2000 commented Mar 11, 2024 •

edited

Loading

dirkf left a comment

gzz2000 commented Mar 12, 2024 •

edited

Loading

dirkf commented Mar 12, 2024

[Vimeo] fix config parser regex #32742

[Vimeo] fix config parser regex #32742

Conversation

gzz2000 commented Mar 11, 2024 • edited Loading

Please follow the guide below

Before submitting a pull request make sure you have:

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

What is the purpose of your pull request?

Description of your pull request and other information

dirkf left a comment

Choose a reason for hiding this comment

gzz2000 commented Mar 12, 2024 • edited Loading

dirkf commented Mar 12, 2024

gzz2000 commented Mar 11, 2024 •

edited

Loading

gzz2000 commented Mar 12, 2024 •

edited

Loading