fix: disable background threads where we use fork

[rgrinberg]

Signed-off-by: Rudi Grinberg me@rgrinberg.com

[rgrinberg]

@anmonteiro just to make sure, did you double check this fix worked?

[emillon]

is the hang on macos CI related to the fix?

[rgrinberg]

Indeed it's likely related. If the fix is indeed what I think it is, only #8090 is a complete fix.

[anmonteiro]

@anmonteiro just to make sure, did you double check this fix worked?

Yes.

[rgrinberg]

@emillon this is good enough for 3.9.1

[emillon]

I'm a bit concerned that this still hangs on macos. Do you understand why that's the case? Do you have an idea if this makes that less likely than on 3.9.0? If we reverted the original feature, would it hang sometimes as well?

[rgrinberg]

I assume it's just because it's not safe to fork in a multi threaded application in OCaml. We still have other uses of threads, and it's not really possible to get rid of them all.

[emillon]

I understand that #8090 will fix that more or less permanently, but it's probably too risky for a backport. So I'm wondering if there's a good way to mitigate what it looks to be a regression (due to the fact we're using more threads than before I suppose)

[rgrinberg]

Thinking a little more. We could mitigate the issue a little more if we:

1. Disable the threaded console on platforms where we use fork.
2. Lazy initialize the Async_io thread. It's only needed in watch mode, so we can disable it for normal builds.

Do you want to give these a try?

[emillon]

Sure! I suppose that the first one is about doing the same as this PR, but for threaded_console below?

[rgrinberg]

Yes, basically.

[emillon]

What about the second one? I see Async_io.with_io in the scheduler but I know nothing about that part of Dune, so tips appreciated :)

[rgrinberg]

Np. At the moment, the various functions in Async_io dispatch the work to the select thread (e.g. ready, close, etc.). Instead of starting the thread eagerly in with_io, we can wait until one of those functions is called before initializing the thread.

[emillon]

I've done that in #8121 and #8122.