Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What is the best number of thread for downloading? #866

Open
2 tasks done
Zykino opened this issue Nov 21, 2017 · 15 comments
Open
2 tasks done

What is the best number of thread for downloading? #866

Zykino opened this issue Nov 21, 2017 · 15 comments
Labels
discussion This needs to be discussed before anything is done downloader Issue is related to the downloader question Not really an issue, but more of a question about how something works

Comments

@Zykino
Copy link

Zykino commented Nov 21, 2017

  • I carefully read the contribution guidelines and agree to them.
  • I checked if the issue/feature exists in the latest version.

As stated in #853 :

I also see a "Number of threads" in the download popup.
With my 4G/WIFI what is the best choice?
When I'm on a slow H+ network (~20kB/s) what should I prefer?
Is it really an option an end user should see? I f I had a cursor "my internet is fast/slow, reliable/unreliable" and let NewPipe do the translation I would be happy enough with it!

For an user the option should be hidden behind an other setting or at least explained with examples:

  • 1 thread if your connection is super slow
  • 3 thread for an average 3G connection
  • Max thread if you are in 4G/LTE

WARNING: I don't how the number of threads impact the download so my examples are likely to be wrong.

@ghost
Copy link

ghost commented Nov 21, 2017

Even a slower phone will do 3 threads just fine. No need to go below 3 unless you have an ancient phone and even that should just have 2 threads. The SOCs on phones, even 5 year old phones are powerful enough to do multi-threaded data transfer with no slowdown at all.

I think most people should keep it at the default 3 unless they are downloading really big Youtube videos. That's where more threads actually comes in handy since there's more data to split into chunks.

@Zykino
Copy link
Author

Zykino commented Nov 22, 2017

Oh so it depend on my phone capability and the video weight ?
I'm sure NewPipe can guess/found infos on the former and has the infos of the later at the start of the download.

With your comment I'm really tempted to put the bar at it's maximum and hope for the best.

@stale

This comment was marked as resolved.

@stale stale bot added the stale label Dec 1, 2018
@ghost
Copy link

ghost commented Jan 25, 2019

Close this issue.

@gkeegan
Copy link
Contributor

gkeegan commented Jan 25, 2019

This isn't exactly fixed. I may try to find optimal thread numbers by downloading a few random files of various lengths using different numbers of threads and timing how long it takes to do it. If I do it I'll make a chart of the data. So please don't close this yet.

@gkeegan
Copy link
Contributor

gkeegan commented Jan 25, 2019

@theScrabi Is there a better way to see how long it took for a download to complete besides comparing the start time to the end time in the logs?

@Stypox
Copy link
Member

Stypox commented Apr 19, 2019

I, too, think the threads option should be hidden and the number of threads should be automatically deduced.
It is just confusing for most users, who don't even know what a thread is. And even if they do, how could they know what the best choice is?

I am willing to implement #2005 and #2123 , but that would require me to add another option to the download dialog. I suggest to remove the threads option alltoghether for the reasons stated above (defaulting it to 3), so that there is more space for that.

@Stypox Stypox added the discussion This needs to be discussed before anything is done label Jun 23, 2019
@TobiGr TobiGr removed the stale label Jul 19, 2019
@Stypox Stypox added question Not really an issue, but more of a question about how something works downloader Issue is related to the downloader labels Jan 27, 2020
@Stypox
Copy link
Member

Stypox commented Jan 27, 2020

@h1nk says in #2946:

From my very minimal understanding, NewPipe currently implements the video, audio and caption download functionality in the app's source itself and not via a library dedicated to downloading YouTube videos? If my further understanding is correct, NewPipe also uses the variable amount of operating system threads that the user sets on a video download task to parallelize the process of downloading different chunks of the desired video. I'm just somewhat confused about the rationale of using operating system threads for download tasks instead of using non-blocking I/O since it's been readily available in Java SE for some time (see #373 (comment)). It just seems like the strict use of threads is a really inefficient way to go about accelerating I/O bound operations like downloading files over the Internet concurrently etc. Is there any specific reasons for this current design? Could this potentially make for a future improvement?

@opusforlife2
Copy link
Collaborator

I, too, think the threads option should be hidden and the number of threads should be automatically deduced.

@Stypox How would that be determined? Simply based on the size of the download?

@Zykino
Copy link
Author

Zykino commented Mar 25, 2020

I mean how, as a user, am I supposed to determine this?
New Pipe have the infos of:

  • file size
  • connection type
  • smartphone specs

The only info I can think of that New Pipe doesn’t know is: will the user move/network conditions change. But even the user cannot be sure of that.

To summarize my thought: If me, a software engineer (not working in network or web) am not sure what this setting is for, how someone bad with tech could change it on a per download basis?

Maybe let the option but burry it in settings, if you prefer to not find an automatic algorithm. Or if you ask peoples to change it sometimes when they have a problem.

@wb9688
Copy link
Contributor

wb9688 commented Mar 25, 2020

We should remove multi-threaded downloading altogether imho, as it doesn't really provide benefits and just comes from the legacy GigaGet stuff our download manager is based on.

@opusforlife2
Copy link
Collaborator

So multi threading isn't useful for any of the services?

@Stypox
Copy link
Member

Stypox commented Mar 25, 2020

Yeah, that's what I wanted to do in my soon-to-arrive download-dialog-redesign PR.

@OROOTALKS
Copy link

Hope This Helps

Number threads per download - the number of simultaneously running threads that independently downloads the pieces of the file. With a certain number of these streams, the apparent increase in the download speed begins to decrease, while the computing load on the computer increases. In other words, there is a threshold after which the speed increase is invisible, but a strong processor load is noticeable. The recommended number is 8-12, but with a significant increase in this number it is recommended to reduce the number of simultaneously active downloads "Maximum active downloads".

@TeamNewPipe TeamNewPipe deleted a comment from ConnieDaniel Apr 4, 2021
@SameenAhnaf SameenAhnaf mentioned this issue Sep 14, 2021
3 tasks
@mase76

This comment was marked as off-topic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion This needs to be discussed before anything is done downloader Issue is related to the downloader question Not really an issue, but more of a question about how something works
Projects
None yet
Development

No branches or pull requests

8 participants