Make task::Builder::spawn* methods fallible

[ipetkov]

Motivation

Tokio's current stable spawn* methods are all infallible but may internally panic on unrecoverable errors (for example, if the blocking threadpool is empty and the OS refuses to spawn additional threads). Unfortunately, this means that applications cannot opt-into gracefully handling such situations.

Solution

Convert all task::Builder::spawn* methods to return a fallible io::Result<_>, giving applications the opportunity to opt-into handling spawn errors themselves. This also is a fitting analogue to std::thread::Builder which also has fallible spawn methods (contrasted with std::thread::spawn which panic on failure).

Given that the task::Builder and its APIs are currently marked as unstable, this is a good time to make the change before we start stabilizing them. Currently spawn_blocking was the only API which I could tell internally panics on some errors instead of yielding them, so it has been updated to surface those errors via the task::Builder::spawn_blocking* APIs.

Note that the behavior of tokio::task::spawn_blocking is maintained as it was before this change:

• invocations will panic if the threadpool has no available threads and the OS cannot spawn additional ones
• if the runtime is shutting down a dummy JoinHandle is returned without panicking.

[huntc]

A question in relation to this: does a JobHandle's is_finished method return of true also signify that a thread has become available given the absence of any other spawn_blocking? I'm hoping so as I can then defer my spawn_blocking if is_finished is false, at least until this fix becomes widely available.

[Darksonn]

@huntc This question is quite confusing to me. What problem do you consider this a fix for?

[huntc]

@huntc This question is quite confusing to me. What problem do you consider this a fix for?

If I can test that a previous task has entirely completed then I’m in a position to spawn a new one. My question is whether this test is reliable. If the spawn method was failable then I wouldn’t need the test. Make sense?

[Darksonn]

No, it doesn't really make sense. The spawn_blocking function would not fail just because there already are tasks, even with this change. The failures have to do with stuff like "runtime has shut down" or "we've hit the OS limit on the number of threads". If there are enough spawn_blocking tasks to fill the thread-pool, then new tasks just go in a queue and they will start when another task exits.

As for what is_finished means, well, it means that the task has finished. Whether this means that a thread is available would depend on whether the queue of new tasks is empty or not.

[huntc]

If there are enough spawn_blocking tasks to fill the thread-pool, then new tasks just go in a queue and they will start when another task exits.

I didn’t appreciate the queue aspect to this. Thanks. In the case of spawn_blocking where I’ve allocated a pool size of two blocking threads, what would be the queue size? Again, I’m looking to rely on determining whether a blocking job has finished before spawning another.

[Darksonn]

The queue of pending spawn_blocking tasks is unbounded. I do not recommend using the limit on the total number of spawn_blocking threads to impose a limit on one particular type of task. If you wish to give it a bound, then I recommend a tokio::sync::Semaphore.

[huntc]

The queue of pending spawn_blocking tasks is unbounded. I do not recommend using the limit on the total number of spawn_blocking threads to impose a limit on one particular type of task. If you wish to give it a bound, then I recommend a tokio::sync::Semaphore.

Thanks for the clarifications. I'm limiting the number of blocking threads to minimise memory usage (running embedded Linux). I'm unsure that a semaphore would assist here as I'm looking for the best indication that a previous task has finished outside of an async context, and so that I can avoid queuing another i.e. I don't want to queue to an unbounded buffer as I'd potentially run out of memory very quickly. Anyhow, you've answered my question by informing me of the unbounded queue so I'm good. I must say though that the queue being unbounded is a surprise. Perhaps separately being able to specify a bound and having the spawn methods fail given the bounds being exceeded would be fine, but then that may be another topic.

[Darksonn]

You can definitely put a limit on the number of tasks using a Semaphore. Its non-async methods works outside of async just fine.

[huntc]

You can definitely put a limit on the number of tasks using a Semaphore. Its non-async methods works outside of async just fine.

I see now that I can use a Semaphore from a non-async context via its try_ methods. However, why would using a semaphore be better than calling the job handle's is_finished method? Thanks again for this continued dialog. I hope it isn't going too off-topic now given that we've already established that there's an unbounded queue and I'd not expect spawn_blocking to fail on exhausting its pool of threads.

[Darksonn]

Unless you only want one task running concurrently, I would find the Semaphore more convenient to use. But they would both work.