Slow extraction speed for Youtube video urls

[dhwz]

• I'm asking a question
• I've looked through the README and FAQ for similar questions
• I've searched the bugtracker for similar questions including closed ones

Question

I'm using youtube-dl only to extract video urls from youtube, it's extremely slow (takes up to 30secs to extract the urls) for a while now. I've also checked the open issues, one was talking about that but the final conclusion was it won't be fixed and yt-dlp could be used. I agree yt-dlp is much faster (takes maybe 3sec to get a URL), any chance we can get such modifications used on yt-dlp also on youtube-dl?
yt-dlp is no alternative for me as I'm fixed to a system which still uses Python 2.7

[dirkf]

Feel free to submit a PR that makes the extraction process faster.

What takes the time now is that YT makes the extractor execute some junk JavaScript to obtain media links that are not throttled, though I took ~6% off this in a recent mod.

The extractor could provide you with undecoded media links but then the download speed will be throttled instead. The current extraction process aims to get the widest combination of unthrottled formats. If a format that is seen in the browser isn't being offered by yt-dl, it seems to provoke complaints.

yt-dlp uses other tactics that are either impractical or beyond the interest/capability of any contributor:

• allowing user control of the YT client for which links are retrieved
• punting to PhantomJS or similar (but see PR [options] Added workaround option to execute "n_function" #31187).

[HPP8140]

any option to prefer throttled links for extraction speed?

[HPP8140]

Not without modifying the code. Also, one part of the JS execution is required when YT sends an encoded media link (signature) to decode the signature, without which you have no media URL, throttled or otherwise.

As always, a full refund of 0 0-based monetary units is offered if you find the program unsatisfactory.

Okay, appreciate the response, thanks for closing the loop on this.

I will keep using the old verson of code, until someting major breaks that I am unable to understand how to manually patch because that part of the code changed so drastically among multiple files LOL. Then I will have to find time to update my SoC to be able to run modern versions of the code on Python 3 etc.

[dirkf]

Recent yt-dl versions are running fine, though slowly, with Python 2.7.1 on a Broadcom 7405 SoC with 2xMIP4380 CPUs (oh, and invoking wget for each _request_webpage()). You just have to adjust expectations.

[HPP8140]

Recent yt-dl versions are running fine, though slowly, with Python 2.7.1 on a Broadcom 7405 SoC with 2xMIP4380 CPUs (oh, and invoking wget for each _request_webpage()). You just have to adjust expectations.

Currently using a legacy PogoPlug E02 SoC running Debian jessie :) Need to upgrade to some current Odroid or Pi

• CPU: Marvell 88F6281
• CPU Cores: 1
• CPU MHz: 1200
• Flash MB: 128
• RAM MB: 256

So instead of <11 seconds for youtube_dl -g, it can take up to 2 minutes per link...so quite a big difference.
After grabbing all links & metadata from my yt watch playlist, I forward links to pyLoad.

[HPP8140]

Recent yt-dl versions are running fine, though slowly, with Python 2.7.1 on a Broadcom 7405 SoC with 2xMIP4380 CPUs (oh, and invoking wget for each _request_webpage()). You just have to adjust expectations.

Well looks like the time has come....it was a good run.
Old Version 2021.12.17 without nsig :) no longer downloading (403)

python -m youtube_dl -v https://www.youtube.com/watch?v=crnDkGUdoew

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=crnDkGUdoew']
WARNING: Assuming --restrict-filenames since file system encoding cannot encode all characters. Set the LC_ALL environment variable to fix this.
[debug] Encodings: locale ANSI_X3.4-1968, fs ANSI_X3.4-1968, out ANSI_X3.4-1968, pref ANSI_X3.4-1968
[debug] youtube-dl version 2021.12.17
[debug] Python version 2.7.16 (CPython) - Linux-3.14.0-kirkwood-tld-1-armv5tel-with-debian-8.10
[debug] exe versions: ffmpeg 4.1.4-1, ffprobe 4.1.4-1
[debug] Proxy map: {}
[youtube] crnDkGUdoew: Downloading webpage
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://rr5---sn-8xgp1vo-p5qy.googlevideo.com/videoplayback?expire=1718503537&ei=EfRtZprqHZ2LkucPq4iggA8&ip=96.255.39.226&id=o-AM3HxxJOOm5Ve13igdogFDC7B84GzF1H3NiNc3ga2Yvn&itag=137&aitags=134%2C136%2C137%2C160&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&mh=KB&mm=31%2C26&mn=sn-8xgp1vo-p5qy%2Csn-ab5l6nkd&ms=au%2Conr&mv=m&mvi=5&pl=18&initcwndbps=1111250&bui=AbKP-1P-XlxvEGAgw9c9nOLxvl0L1MW7_fKyxhf-bG6kdTsfgejTfqaDcx2RgNH9wcp2M5qM8R0m_RSG&spc=UWF9f2TkcOed3XvUROmKS8L34hfTstUwPV-IcAlM0I9URngKM3s8vcHmlgMi&vprv=1&svpuc=1&mime=video%2Fmp4&ns=3kshX-WwBaduvXJu2wKIW-4Q&rqh=1&gir=yes&clen=843238223&dur=1563.366&lmt=1718471588148623&mt=1718481655&fvip=5&keepalive=yes&c=WEB&sefc=1&txp=7209224&n=ubc0lAQ-fTeNojJB&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRAIgAyYWfgDeNSuJk6zwKp2_I3fJWW9scii8oFV6aMp-D5MCICfSI0fCMCtIDzh59Mpl5r3et8LxH7qCvrRxik8cIilY&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AHlkHjAwRQIgFDmUYPolqFMl-anaT9e67R1pdDyL3zZtDAwR_LLmMMoCIQC6XqWFIvp1yr_jenzMuHBw-0CMf4jaiIV0909FK6mGcg%3D%3D'
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "/usr/lib/python2.7/youtube_dl/YoutubeDL.py", line 1970, in process_info
    partial_success = dl(fname, new_info)
  File "/usr/lib/python2.7/youtube_dl/YoutubeDL.py", line 1915, in dl
    return fd.download(name, info)
  File "/usr/lib/python2.7/youtube_dl/downloader/common.py", line 366, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python2.7/youtube_dl/downloader/http.py", line 351, in real_download
    establish_connection()
  File "/usr/lib/python2.7/youtube_dl/downloader/http.py", line 116, in establish_connection
    raise err
HTTPError: HTTP Error 403: Forbidden

[dirkf]

Use the nightly build (#30839, Known Issues) and report if that fails. The old release has probably not worked in many cases for several years.

[HPP8140]

Thanks yeah nightly build works, but then stopped working yesterday, but now looks like it's working again. Need to hurry up and set-up a faster SOC.

[dirkf]

This broke 2021.12.17. The nightly build was working until YT player 590f65a6 and then started working again when that player was replaced by 84314bef.