scoped threads vs thread leaks

[RalfJung]

The standard library scoped threads do not use the OS facilities for joining threads, but instead they implement join logic with their own synchronization primitives. (This seems to help with cleaning up thread resources ASAP rather than keeping them around until the scope finishes and all threads are joined.)

Unfortunately this can show up in Miri as a thread leak, if a thread gets scheduled away just after the custom join logic is done, and then the main thread finishes.

For now, a possible work-around is to use -Zmiri-ignore-leaks. But of course it'd be nice to find a better solution. Maybe, after the main thread is done, instead of halting execution immediately we could yield that thread N times (for some suitable N), thus giving other threads a chance to finish?

[oli-obk]

As an immediate step we should probably add a test for it ^^

The "leaked" threads don't have anything interesting to finish doing, so maybe we could have the scoped cleanup logic mark them as leakable or just join them under cfg(miri)

[RalfJung]

Yeah I considered just adding this to the standard library somewhere after main returns:

if cfg!(miri) {
  // Give other threads a chance to finish their work.
  for _ in 0..100 { thread::yield_now(); }
}

But honestly that seemed a bit too hacky.^^ Seems better to do this on the interpreter side.

[oli-obk]

No I meant... actually join the threads in scoped thread code instead of just cleaning up their resources and letting them run to completion.

[RalfJung]

That would be a much bigger and more annoying set of cfg(miri).

[saethlin]

Do we have a reproducer for this? It sounds to me like a really tight window?

[RalfJung]

Yes it's a tight window. I definitely saw this happen, which is how I even noticed, but don't remember the exact code. But a test will always risk false negatives here.

[RalfJung]

For example, this code

use std::thread;

fn main() {
  thread::scope(|s| {
    s.spawn(|| {});
  });
}

reproduces the problem with seed 9.

But you usually need to try a bunch of seeds to see this happen.

[saethlin]

Is there any particular reason to believe that "yield to all other threads on thread exit" isn't something we should/can do in the interpreter instead?

[RalfJung]

That's exactly what I suggested in the OP. :)