Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Under new management #30568

Closed
dirkf opened this issue Jan 29, 2022 · 173 comments
Closed

Under new management #30568

dirkf opened this issue Jan 29, 2022 · 173 comments

Comments

@dirkf
Copy link
Contributor

dirkf commented Jan 29, 2022

Thanks to @rg3 who created this program the project has a new maintainer.

Also, many thanks to @dstftw and @remitamine for holding the fort over the last several years.

I hope that we'll be able to make a new release soon and subsequently keep the program more up-to-date than has been the case for the last few months.

The project has a fork https://github.com/yt-dlp/yt-dlp that offers a lot of extra functions but demands an up-to-date Python version. This project will continue to target Python version 2.6, 2.7, or 3.2+, at least until no-one complains about 2.6 compatibility.

PRs are very welcome, although there is a significant back-log to be handled. Back-ports of yt-dlp features are also welcome.

Finally, I'd encourage anyone else who is interested in sharing maintenance duties to establish a track record and make themselves known. We want to keep this popular project alive with a community of future maintainers.

@Twangist
Copy link

Supporting defunct versions of Python seems like a great way to handicap the project and a total waste of time.

@boehs
Copy link

boehs commented Jan 29, 2022

This repository should just become yt-dlp tbh

@VixieTSQ
Copy link

this is ridiculous. Python 2 is dead

@pukkandan
Copy link
Contributor

This repository should just become yt-dlp tbh

By now, yt-dlp has accumulated various incompatibilities with youtube-dl and it is no longer a drop-in replacement for many users. So youtube-dl can't just "become yt-dlp". Since I do not intend to revert these changes, it is best that both projects co-exist.

If you like to use yt-dlp, use it. And if you prefer youtube-dl, use this. It is a waste of everyone's time to post issues/comments asking for either project to "become" the other

@ligix
Copy link

ligix commented Jan 29, 2022

fyi python 2 reached EOL 2 years ago, before windows 7 did.

https://www.python.org/doc/sunset-python-2/

@boehs
Copy link

boehs commented Jan 29, 2022

This repository should just become yt-dlp tbh

By now, yt-dlp has accumulated various incompatibilities with youtube-dl and it is no longer a drop-in replacement for many users. So youtube-dl can't just "become yt-dlp". Since I do not intend to revert these changes, it is best that both projects co-exist.

If you like to use yt-dlp, use it. And if you prefer youtube-dl, use this. It is a waste of everyone's time to post issues/comments asking for either project to "become" the other

fair enough! Consider mentioning dlp in the readme though?

@dirkf
Copy link
Contributor Author

dirkf commented Jan 30, 2022

@Twangist and supporters, please direct your Python 3.7+ enthusiasm to the yt-dlp project, as @pukkandan says.

No-one would bother maintaining two versions of a program if there wasn't a good reason. There are still applications where older Python versions have to be supported: among the known cases are those where that's all the platform offers, and where yt-dl is being embedded in a larger chunk of Python 2 code that no-one is going to port.

This project has already established a compatibility library that largely allows you to code for Python 3 while remaining compatible with Python 2, by using the library's compat_xxx types and avoiding novel syntax elements (nonlocal, yield from, etc), for which there are known work-arounds. With this infrastructure, and provided that yt-dl and yt-dlp continue to support the same extractor API and library functions we can avoid wasted effort on either side.

The project Readme should definitely be updated, in due course, as @boehs suggests.

@dirkf dirkf closed this as completed Jan 30, 2022
@dirkf dirkf reopened this Jan 30, 2022
@chrizilla
Copy link

#30568 (comment) :
The project has a fork https://github.com/yt-dlp that offers a lot of extra functions
Back-ports of yt-dlp features are also welcome.

#30568 (comment)
This repository should just become yt-dlp tbh

Wouldn't merging the two projects be a better use of resources
compared to the overhead/redundancy of maintaining two separate projects ?

@dirkf
Copy link
Contributor Author

dirkf commented Jan 30, 2022

Essentially #30568 (comment) said all that needed to be said. But...

Understandably, yt-dlp wants to be able to rely on supported Python versions (and 3.6), which means that we need to keep yt-dl up-to-date to address the cases I described above that can't practically use those versions, desirable as it might be.

For people concerned that Python 2 is a couple of years beyond EOL, one case that I support could survive until 2038, or the end of DVB-T2 broadcasting if that comes sooner. That's the timescale of deployed embedded applications, a very different world from 'What version is Chrome today?' on your personal computing device.

Maybe some future Python version supported by yt-dlp will diverge so far that common development becomes impractical. This isn't the case now.

@dimitarvp
Copy link

Appreciate the openness, @dirkf. I personally wasn't aware that one of youtube-dl's goals was Python 2 compatibility so thanks for clearing that up for us.

I am not and will not ever advocate for projects closely mirroring each other. But, in case that's a useful user feedback for you: my main reason for moving to yt-dlp was (a) SponsorBlock API support and (b) external downloaders. If these sound like an interesting thing to support for you then I believe others would love to see them implemented as well. If not, the project is still super valuable in its own right.

Thanks for the update. It's appreciated.

@spectrapulse
Copy link

Wouldn't it be better to either release an LTS or to set a date when support for everything below 3.6 is dropped.

Motivating people to stay on EOL versions of Python carries a lot of security risks, not just for this project but also for projects that depend on it.

And I would assume that older projects would most likely already have their dependencies version locked anyways.

@Evernow
Copy link

Evernow commented Jan 30, 2022

Python 2 is end of life and no longer receiving any security updates, you supporting Python 2 will just motivate others to not move on from insecure unmaintained dependencies.

You'll also be splitting up manpower by not just encouraging use of yt-dlp, which will affect everyone, not just those who insist on not realizing the past is the past.

@gamer191
Copy link

You'll also be splitting up manpower by not just encouraging use of yt-dlp, which will affect everyone, not just those who insist on not realizing the past is the past.

To be fair, yt-dlp usually implements all of youtube-dl's commits pretty quickly (unless they conflict with yt-dlp), so youtube-dl continuing to be maintained won't negatively affect yt-dlp that much. Also, #30568 (comment) said that youtube-dl's readme will be updated to mention yt-dlp

@Earnestly
Copy link

Who is @dirkf?

@dirkf
Copy link
Contributor Author

dirkf commented Jan 30, 2022

Just your standard Internet hound.

Just to confirm, Python 2 (or 2 and < 3.6) compatibility wasn't originally a goal of the project since Python 3 appeared only after the project was created, but that level of compatibility is a goal now, since yt-dlp exists for more recent Python versions. And anyone who can practically do so should run a supported Python version, on which yt-dl should run equally well.

@gamer191
Copy link

gamer191 commented Jan 30, 2022

The project Readme should definitely be updated, in due course, as boehs suggests.

May I suggest going one step further and informing the user about yt-dlp every time they run youtube-dl --update, if youtube-dl detects that it's running on python 3.6 or above. The user would have to also be informed about the slight differences between youtube-dl and yt-dlp, and that if they update they need to move their config file. Obviously this would be controversial and have downsides, one of which is that it may result in yt-dlp users accidentally reporting issues on the youtube-dl repository.

EDIT: Sorry boehs for the ping, it was an accident

@Lesmiscore
Copy link
Contributor

I'd suggest you to unpin (or move to the lowest) #27013 because its title is a bit confusing.

Anyway, I'd appreciate youtube-dl now have a new maintainer.

@dirkf
Copy link
Contributor Author

dirkf commented Jan 30, 2022

@gamer191 wrote:

May I suggest going one step further and informing the user about yt-dlp every time they run youtube-dl --update,...

You can suggest it, but I think just updating the documentation will be quite enough. Feel free to try your luck with a PR, though, but I don't fancy your chances.

@Lesmiscore wrote:

I'd suggest you to unpin (or move to the lowest) #27013 because its title is a bit confusing.

Good point, it's not exactly news now, and I hope it won't be again.

@nocturn9x
Copy link

nocturn9x commented Jan 30, 2022

Stubbornly supporting products that are way past their due date is absurd. Stop poking at dead bodies and embrace change rather than avoid it at all cost.

Do what the CPython team did: grow some balls, deprecate and move on. People will be angry, it always happens. But enough people are angry now with this project supporting a Python release that is 12 years old.

It's not like deprecated software suddendly stops working. It just isn't maintained anymore: if anyone still needs Python 2.7 support it's either their fault or their problem to deal with and certainly not a good reason to stifle the projects' growth, creating confusion and splitting manpower across 2 projects doing basically the same thing

@gamer191
Copy link

sorry, I forgot about mentioned this issue notifications

@noahbroyles
Copy link

Just your standard Internet hound.

Just to confirm, Python 2 (or 2 and < 3.6) compatibility wasn't originally a goal of the project since Python 3 appeared only after the project was created, but that level of compatibility is a goal now, since yt-dlp exists for more recent Python versions. And anyone who can practically do so should run a supported Python version, on which yt-dl should run equally well.

So is the only point of even keeping this project alive to support hella old versions of Python? Is any new functionality gonna be added? I'd love to see a few issues be closed on this that would actually improve the project. This is the OG of yt-dlp, after all. Let's not let the clone become better than the OG.

@EvanCarroll
Copy link

This is the OG of yt-dlp, after all. Let's not let the clone become better than the OG.

If you could think of any way to quantify what would signify the point in which "the clone" became better, you'd have already realize that point is at least 3 years in the past. Welcome.

@noahbroyles
Copy link

If you could think of any way to quantify what would signify the point in which "the clone" became better, you'd have already realize that point is at least 3 years in the past. Welcome.

@EvanCarroll yeah, I stopped using youtube-dl and switched to yt-dlp because of the damn n descramble blowup. What I meant to say is, we have some catching up to do.

@Tachi107
Copy link

C is 50 years old, and the device you're reading this on relies on it for all of its basic functionalities... I'm pretty sure that youtube-dl can continue to be developed while targeting Python 2.6 :)

@EvanCarroll
Copy link

EvanCarroll commented Sep 20, 2022

@Tachi107 That's silly. You're arbitrarily looking at the first-release date, which has no relevance to the conversation.

  • C - the language - is still being maintained (ie C17)
  • C - the compilers - are still being maintained GCC / Clang, in the open source world. New versions come out quite frequently.
  • C - standard libraries are maintained - MUSL, libc, etc.

None of this is true for Python2. It's dead.

@Tachi107
Copy link

Tachi107 commented Sep 20, 2022

You're arbitrarily looking at the first-release date, which has no relevance to the conversation.

That's fair, but it is also true that new C versions are not widely available (MSVC does not even completely support C11...). And in any case, projects like curl still use C89, because regardless of what the language is doing, if you want to stay highly portable you have to stick to outdated stuff, see the next point.

None of this is true for Python2. It's dead.

Who cares. If it has users, you have to respect the project's decision to continue supporting it. Believe it or not, some platforms are still stuck with Python2 (like NonStop OS on ia64), and do not have any Python3 port. Yeah, you and I probably don't need to worry about this, but somebody else might do :)

Edit: please, don't go to the OpenSSL thread to ask things unrelated to their issue, you'll notify all the people subscribed to that thread.

@EvanCarroll
Copy link

EvanCarroll commented Sep 20, 2022

It doesn't have users, and you're just wrong on those points. =)

  • What MSVC does is of no consequence., but it does support C11, and C17:

  • C11/C17 is highly portable, regardless of what MSVC does. There is a C17 compiler for like every platform produced: https://gcc.gnu.org/install/specific.html

  • Curl doesn't use C89, for portability. They use it because that's what they did in the 90s. Most C programs have since at least updated to C99. From the FAQ,

    We started out using C89 in the 1990s because that was the only way to write a truly portable C program and have it run as widely as possible. C89 was for a long time even necessary to make things work on otherwise considered modern platforms such as Windows. Today, we do not really know how many users that still require the use of a C89 compiler.

    I can't see "we do not really know how many users that still require the use" substantiating the claim that they need to use C89 for portability. But reminder, C89is still a supported language unlike Python2. You can still compile programs in every modern C compiler using the C89 standard. And there is still a support ecosystem and toolchain to ensure your code is valid C89. That is to say, it's and secure and maintained this is just a rehash of your prior argument about looking at "arbitrarily looking at the first-release date".

Who cares. If it has users, [...] "continue supporting it"

It doesn't, and if did they're running insecure code. But if there were users and if Python 2 was supported upstream, I would agree.

@Tachi107
Copy link

This thread is not about C (but you're free to try to use C11 threads on Windows, regardless of which compiler you use - it is not going to work).

Who cares. If it has users, [...] "continue supporting it"

It doesn't, and if did they're running insecure code. But if there were users and if Python 2 was supported upstream, I would agree.

Security is not that relevant in all contexts (and is overlooked when it really matters), and, as already mentioned in this thread, there're plenty of embedded (and not) systems running a fixed release of python; my local train station still uses Debian 6 or something, for instance.

Anyway, you're free to think and use whatever you want - just stop being so ungrateful towards people developing software you use for free.

@dirkf
Copy link
Contributor Author

dirkf commented Sep 20, 2022

Indeed, #30568 (comment) (there are other cases).

I'm quite sure the users of the STB application would rather use "insecure code" to access YT and BBC catch-up content that the STB used to support than not be able to do so at all. Equally, they aren't going to be upgrading from Linux kernel 2.6. A Python 2.7 system will still run correct and diagnose incorrect Python 2.7 programs regardless of the support status designated by PSF.

The linked OpenSSL thread is interesting in that **** build tool dependencies as discussed there are exactly why no Py3 exists on the STB platform (sizing might also be a problem, but was never reached).

@scrutinizer11

This comment was marked as off-topic.

@a-pav
Copy link

a-pav commented Mar 14, 2023

I wish you could just merge the two projects, I think that would have been for the best.

But at least both projects should look alive and have regular releases in order to not lose community focus. I personally don't care for the new shiny features of yt-dlp. I prefer to use the good old widely known youtube-dl but that is getting harder and harder, considering the recent failures (due to changes on the youtube side). It seems that most of the new contributions are happening on yt-dlp, while most new users/adapters are, most probably, searching for youtube-dl and disregard anything else. Beside that it also seems that most of the contributors don't share the vision of the leads here in regards of keeping support for Python 2 and are already moving (moved) away.

I haven't seen anyone asking you to keep supporting Python 2 but I see people asking for a new youtube-dl release left and right.

@dirkf dirkf mentioned this issue Mar 16, 2023
3 tasks
jmintb added a commit to jmintb/WLASL that referenced this issue Mar 18, 2023
./start_kit/video_downloader.py was not working with some youtube
videos, due to youtube-dl out of date.

Therefore this commit updates ./start_kit/video_downloader.py to use yt-dlp which is
a more up to date fork of youtube-dl.

It looks like youtube-dl is under new maintenance and may become
viable in the future. Therefore the youtube_downloader variable can be
changes in ./start_kit/video_downloader.py to easily switch between the two.

Relevant github issue for youtube-dl status: ytdl-org/youtube-dl#30568
jmintb added a commit to jmintb/WLASL that referenced this issue Mar 18, 2023
Addresses: dxli94#56

./start_kit/video_downloader.py was not working with some youtube
videos, due to youtube-dl out of date.

Therefore this commit updates ./start_kit/video_downloader.py to use yt-dlp which is
a more up to date fork of youtube-dl.

It looks like youtube-dl is under new maintenance and may become
viable in the future. Therefore the youtube_downloader variable can be
changes in ./start_kit/video_downloader.py to easily switch between the two.

Relevant github issue for youtube-dl status: ytdl-org/youtube-dl#30568
jmintb added a commit to jmintb/WLASL that referenced this issue Mar 18, 2023
Addresses: dxli94#56

./start_kit/video_downloader.py was not working with some youtube
videos, due to youtube-dl out of date.

Therefore this commit updates ./start_kit/video_downloader.py to use yt-dlp which is
a more up to date fork of youtube-dl.

It looks like youtube-dl is under new maintenance and may become
viable in the future. Therefore the youtube_downloader variable can be
changes in ./start_kit/video_downloader.py to easily switch between the two.

Relevant github issue for youtube-dl status: ytdl-org/youtube-dl#30568
@K4sum1
Copy link

K4sum1 commented Apr 4, 2023

So many fucking people complaining about a good thing. More support means more users. A basic fucking video downloader should work on a toaster, not require the latest """""""""secure""""""""" version of x software that has years of extra bloat and tells you to fuck off if your system is barely out of date.

@EvanCarroll
Copy link

@K4sum1 yt-dlp works on python3.7+, hardly new. It was released in 2018. It's the oldest still supported version of Python 3.

@Twangist

This comment was marked as abuse.

@dirkf
Copy link
Contributor Author

dirkf commented Apr 5, 2023

Apparently there are plenty of users who want youtube-dl rather than yt-dlp, for whatever reasons, and complain when the latter is suggested as an alternative. This was demonstrated when YT broke the program. At least one user wanted yt-dl running on Python 2.6 (it does, with some limitations).

I haven't seen anyone asking you to keep supporting Python 2 but I see people asking for a new youtube-dl release left and right.

The first (and see above) doesn't require anything to be changed; the second requires new work.

Despite some 75 PRs being merged since this issue was created, no PRs have failed because of Python language issues. There has been quite full co-operation and cross-fertilisation between this project and yt-dlp, including straightforward back-ports of some yt-dlp features coded for Py3.7+.

This issue is no longer pinned and has been quite fully discussed. To avoid issue necromancy, I'm going to close and lock it now.

@dirkf dirkf closed this as completed Apr 5, 2023
@ytdl-org ytdl-org locked as resolved and limited conversation to collaborators Apr 5, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests