Nested install() has counter-intuitive behavior

[zopieux]

Hello,

Please refer to this Playground link.

// Pseudo code. See Playground link.
cheap_pool = ThreadPoolBuild::num_threads(12);
heavy_pool = ThreadPoolBuild::num_threads(2);
fn cheap() { /* cheap work here */ }
fn heavy() { cheap_pool.install(|| { (0..1000).into_par_iter().map(cheap).collect() }) }
fn main()  { heavy_pool.install(|| { (0..10).into_par_iter().map(heavy).collect() }) }

My main goal here is to split rayon's workload in two categories:

• heavy work, which in real life brings up heavy resources and should have limited concurrency, say 2 concurrent workers.
• cheap work, represented by a sleep in my Playground. This should run with high concurrency (I've hardcoded thread count to 12 just to illustrate the issue).

Thing is, each heavy work item depends on calling many cheap work items. In my example I need to process 10 heavy items, each spawning 1000 cheap items.

Since it seems like rayon has no API to limit worker count per par_iter invocation (#826 maybe), I've instead tried to create a pool for heavy work, a pool for cheap work, and nesting them through the use of pool.install which according to docs, makes par_iter use that pool.

The code prints a millisecond timestamp so we can easily debug how many items are worked on in parallel.

What I thought the code would do

Show lines with similar timestamps in batches of 2, since I've limited to 2 threads, and each takes roughly the same amount of time (1000 short sleeps spread across 12 threads):

[0] item 0, from thread 0
[0] item 4, from thread 1
… some time …
[4001] item 1, from thread 0
[4002] item 5, from thread 1
… some time …
etc.

What the code actually does

9 heavy work items are immediately scheduled concurrently (even though they're inside a 2-thread pool install), then all the 9000 sleep go to work, then the last work item starts (1000 sleeps), then the program finishes.

[0] item 0, from thread 0
[0] item 5, from thread 1
[1] item 2, from thread 0
[1] item 6, from thread 1
[1] item 3, from thread 0
[1] item 7, from thread 1
[1] item 8, from thread 1
[1] item 9, from thread 1
[1] item 4, from thread 1
[4001] item 1, from thread 0

It's as if the outer (heavy) par_iter pool somehow gets taken over by the nested (cheap) par_iter pool and therefore schedules the heavy work on it.

Am I misunderstanding the install API? What should I do differently? Is that a bug in rayon?

Thanks for your help!

[zopieux]

Ah crap, could it be #765?

[cuviper]

Am I misunderstanding the install API?

I think you're missing what happens to the caller while it waits for the installed task to complete in the other pool. If you call install from a non-rayon thread, it simply blocks until completion. But when called from one pool to another, the caller won't actually block, but rather will participate in work stealing in its own pool.

could it be #765?

Yeah, pretty much.

[zopieux]

Thanks, are you saying that this use-case cannot currently be expressed nicely in rayon and I need to wait for a new primitive/API to be introduced?

[cuviper]

I think something like the "critical section" idea would make this pretty nice and easy, but there are probably ways you can hack around that in your own code. For example, you could use a scope with a precise number of spawns for your heavy pool, with each spawn pulling from your own queue of work -- then that would not be visible to work stealing, and the thread will simply sleep while it waits for the other pool.

[zopieux]

I see, will have a try. Closing as this is virtually a duplicate of ongoing work/discussions.