This is a great project. Can you provide a tutorial to introduce the architecture design of this project?

[webhwh]

This is a great project. Can you provide a tutorial to introduce the architecture design of this project?

[sylviiu]

I've never done one of those before (I'm still new to public open source projects, it's all very new to me lol)

What exactly would the tutorial entail?

[webhwh]

haha sigh。。。
actually I want to know why yt-dlp starts so fast. I used to use Electron to start yt-dlp very slowly. My English is poor, and I am not good at Electron, so it is difficult for me to understand your code without any notes...
If there are any comments or video tutorials explaining the code architecture, I would greatly appreciate it..
Supplement This project is too cool. Thank you for your open source efforts

[sylviiu]

Ah! I apologize for my incredibly messy code lol, I need to make commenting a habit.

It's actually not an optimization in Node.JS / Electron that made yt-dlp start very fast; it was actually a Python bridge I wrote for yt-dlp; the method I use is somewhat "hacky," but it definitely had a good turnout.

Each time you execute the yt-dlp binary, it has to load all of the relevant Python libraries on each execution (which at least on Windows systems I've tested, made it drastically slower); I wanted to figure out another way to make that more optimized.

The Python bridge's main purpose is to allow the executable to be ran once (and continue running until closed), and basically provide a "constantly running yt-dlp launcher" -- the goal of it was to load all of the libraries required by yt-dlp during startup, so every yt-dlp command can be executed instantly by that running process without slowing down your computer with loading all of the libraries needed each time a command was run.

While it was in early development, I made a little comparison between using standard yt-dlp (left side, v1.0) and the Python bridge (right side, v2.0+)

pythonws.1.mp4

downloading_playlist.1.mp4

The bridge also implements Python's threading system, so each command that is executed will spawn asynchronously while maintaining only one running process -- 12 downloads in the app would've spawned 12 different processes if it were using yt-dlp itself, whereas it keeps a single process using the bridge.

[webhwh]

thank you for helping me， you are so kind
no wonder I don't understand. I haven't used Python before -- lol,
a very impressive Python bridge, I guess it might be faster to use Java or C++for the bridge?
although I don't even know lol

[sylviiu]

a very impressive Python bridge, I guess it might be faster to use Java or C++for the bridge?
although I don't even know lol

It could be faster, but I chose Python because yt-dlp is also written in Python - there is an entire Python API exposed by yt-dlp (and upstream) for developers to integrate, which also simplified the library loading.

(and in my testing, even with 7 concurrent downloads, the bridge doesn't use many resources; Python seems to be working perfectly in terms of speed :D)

[webhwh]

ohh, very cooooool ... continuous attention

[GunGunGun]

a very impressive Python bridge, I guess it might be faster to use Java or C++for the bridge?
although I don't even know lol

It could be faster, but I chose Python because yt-dlp is also written in Python - there is an entire Python API exposed by yt-dlp (and upstream) for developers to integrate, which also simplified the library loading.

(and in my testing, even with 7 concurrent downloads, the bridge doesn't use many resources; Python seems to be working perfectly in terms of speed :D)

I think it's best to submit your patch to yt-dlp repo, because if this get integrate into yt-dlp it's a win-win for us, so much improvement can be made from using this bridge honestly, mpv and other project that makes use of yt-dlp also benefit from it which is amazing!

[sylviiu]

I think it's best to submit your patch to yt-dlp repo, because if this get integrate into yt-dlp it's a win-win for us

Not sure that's a goal for them; the bridge is functionally a wrapper for yt-dlp, and being as niche of a use case it is I'd think it's best to keep them separate. I don't intend to submit a patch myself, if I'm going to be honest.

However, I don't mind submitting a patch with a similar function to this if a contributor of yt-dlp says it'd be a good addition beforehand, but again, I don't think it's one of their goals.

[GunGunGun]

I think it's best to submit your patch to yt-dlp repo, because if this get integrate into yt-dlp it's a win-win for us

Not sure that's a goal for them; the bridge is functionally a wrapper for yt-dlp, and being as niche of a use case it is I'd think it's best to keep them separate. I don't intend to submit a patch myself, if I'm going to be honest.

However, I don't mind submitting a patch with a similar function to this if a contributor of yt-dlp says it'd be a good addition beforehand, but again, I don't think it's one of their goals.

We should try to discuss with @pukkandan, honestly in practice being able to run yt-dlp as system service is such a big deal because it eliminates slow startup time, and if we make use of requests/urllib3's ConnectionPool we gain even more speed, this is huge for media players with yt-dlp backend like mpv.

[pukkandan]

yt-dlp is primarily a CLI tool, not a service/daemon and we provide a Python API for extendability. This project is basically a JRPC wrapper for a persistent yt-dlp and has a completely different user interface. It makes zero sense for this to part of yt-dlp itself.