Jupyter Notebook multithreading

[lzxnl]

I'm running Jupyter Notebooks on Julia 1.2 and if I use Base.Threads, set Threads.nthreads() = 12, confirm that it's 12, and then run something simple like
for i in 1:100
println(Threads.threadid())
end
I always get a long stream of 1s. Any attempt to multithread results in no change to performance as only one thread is ever in use. However, running the same code in Juno/Atom does print out different threadids. Does anyone have ideas?

[twavv]

There's no way to start more Julia threads at runtime, unfortunately. Beyond that,

Threads.nthread() = 12

is not really idiomatic Julia. All it does is overwrite the function that calls into Julia's internals to determine the number of threads so that it just returns the number 12 - it doesn't actually increase the number of threads in any way. Typically, the kinds of APIs that do that kind of thing will look like Threads.set_nthreads!(12) (but, as I mentioned, that doesn't exist in Julia).

If you want to do multithreading in Julia with Jupyter, you have to create a new kernelspec.

using IJulia
IJulia.installkernel("Julia 12 Threads", env=Dict(
    "JULIA_NUM_THREADS" => "12",
))

Then you can create a new notebook using that kernel (or change the kernel of an existing notebook in the Kernel > Change Kernel menu) and run code as you expect.

Threads.@threads for i in 1:10
    println(Threads.threadid())
end

outputs (on my machine)

5
3
6
4
8
7
2
9
1
10

[stevengj]

We should update the installkernel command to provide a way to set custom environment variables...

[twavv]

Also, #854 has kind of been left to rot a bit and could be solved at the same time.

[stevengj]

#854 is orthogonal to adding support for specifying environment vars, and combining unrelated changes into a single PR is not a good way to speed things up.

[twavv]

Fair enough. Both are related to installkernel but the actual features don't depend on each-other.

[nilshg]

This can be closed - installkernel allows for adding kernels with multiple threads.