Clarify the behavior of @threads for

[tkf]

The introduction of the new scheduler for @threads #43919 triggered some discussions:

• Add a :dynamic scheduling option for Threads.@threads #43919 (comment)
• https://discourse.julialang.org/t/behavior-of-threads-threads-for-loop/76042/17

which made me realize that the current docstring does not explain what it does clearly and accurately. So, I tried to clarify the docstring and also define the properties of the :dynamic scheduler a bit more to help users understand when to use it. Along the way, I also re-organized the docstring to hopefully make it easy to navigate.

ping @carstenbauer @Moelf

[vchuravy]

Not sure the reference to set_num_threads is helpful?

[tkf]

Yeah, I thought it was weird but I assumed someone wanted to add it. But looking at "git blame", it was added in #38606 which was a rather very broad "catch-all" patch. So I think dropping here makes sense.

@mcabbott Do you have some particular reasons to keep it?

One way to make it relevant is to link to @carstenbauer's https://carstenbauer.github.io/ThreadPinning.jl/dev/explanations/blas/ or add something similar to the manual.

[mcabbott]

No strong feelings. The intention was to point out that what's inside your loop may already be multi-threaded, and that you may want to disable that. But perhaps just a link doesn't help much unless you already know? Something like a manual section might be clearer.

[carstenbauer]

@tkf I think that something along the lines of what I wrote there should, in some form, go into the manual. AFAIK, this is also @ViralBShah's opinion (see here).

[tkf]

OK, I opened issue #44201 for tracking this. We can add a link once we add a manual section.

[IanButterworth]

Reading this, I am again wondering whether :dynamic is the right name..

@tkf your :uniform suggestion is making more sense now that this new behavior is default. i.e. if it wasn't default, I think the name isn't evocative enough, but as default it doesn't need to be.

So thinking ahead:

• :static - legacy, and makes sense
• :uniform - the new (default) behavior, for uniform division of work
• :dynamic - reserve for potential future load-balanced option?

[tkf]

I prefer to focus on documentation in this PR, although I tried to structure it such that adding a new scheduler is easy. I'm not planning to do it myself, but I think we can make :dynamic more dynamic in a separate PR.

[tkf]

"typically less than 10 microseconds" is from

julia> Threads.nthreads()
2

julia> @benchmark wait(Threads.@spawn nothing)
BenchmarkTools.Trial: 10000 samples with 4 evaluations.
 Range (min … max):  1.455 μs … 53.389 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     9.047 μs              ┊ GC (median):    0.00%
 Time  (mean ± σ):   9.004 μs ±  1.242 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

                                        ██▁   ▁▄▄▂
  ▂▁▁▂▁▁▁▁▂▁▂▁▁▁▂▁▂▂▁▁▁▂▂▂▂▂▂▂▂▂▃▃▄▃▆▆▄▆███▆▅▆█████▆▅▄▄▅▅▅▄▃ ▃
  1.46 μs        Histogram: frequency by time        11.5 μs <

 Memory estimate: 480 bytes, allocs estimate: 5.

I'm looking at the median (and the mean, which is not very different from the median) as a "typical" case. It's more useful than min since the multi-queue is a random and concurrent algorithm.

This is on Linux. Can someone check it with other OSes? Note that it's important to use nthreads() > 1 since nthreads() == 1 is special-cased.

[lassepe]

Shouldn't the second part of the statement be reversed?

and the run-time of f(x) is relatively smaller larger than the cost of spawning and synchronizing a task (typically less than 10 microseconds).

[Moelf]

no, it's correct as it is. If the cost of f(x) is larger, then the cost of @spawn would be negligible, so you won't see @threads :dynamic for being more efficient.

[KristofferC]

Bump, can this be merged?

[oscardssmith]

LGTM