A shim mode for targets without threading support

[RReverser]

Maybe I haven't looked through issues closely enough, so apologies if this was brought up before.

One thing that comes up often when using Rayon is making sure that code works correctly on targets without threading support (e.g. wasm32-wasi or regular wasm32-unknown-unknown without threads support in general).

Probably the most common way to do this in a way that abstracts away platform differences is to build custom shim for the used rayon APIs that uses regular Iterator instead and expose it as mod rayon that will be included instead of rayon crate when corresponding feature is disabled (e.g. 1, 2, 3, 4).

However, Rayon itself feels like a lot more appropriate place for such shimming, since it has more control over APIs and would be easier to maintain than custom shims in each separate app that wants to build Rayon-dependant code in a cross-platform manner.

So, the question is: how realistic would it be to make Rayon use regular Iterators when a corresponding Cargo feature is enabled? Given that Rayon can already handle RAYON_NUM_THREADS=1 case, and the behaviour should be pretty similar, it seems doable but I'm not familiar with internals enough to estimate amount of work this would take.

[nikomatsakis]

Hmm. Interesting question. How much does it matter if the code runs fast? I think the question is where is the right place to do this -- I think you could do it in a pretty narrow way at the right spot, but the resulting code might have some extra operations here and there.

[cuviper]

spawn is a particular difficulty. I suppose we might force that to run synchronously, but code may expect independent progress.

Another way to look at that: even with RAYON_NUM_THREADS=1, there are usually at least two threads -- main and the single-threaded pool.

[cuviper]

See also #591.

[nikomatsakis]

Mm, yes, that's a good point.

[cuviper]

AIUI, the standard library for wasm still builds as if it has threading support, but it panics at runtime. So one possibility for that case is to use my rayon-cond wrapper for iterators, and always choose the serial mode for wasm builds. As long as you're consistent with that, it won't ever try (and fail) to create the actual thread pool.

The other shims you linked are ignoring the fact that there are real API differences between Iterator and ParallelIterator. It must be close enough for their needs, but fold is a big example where they're not interchangeable.

[RReverser]

The other shims you linked are ignoring the fact that there are real API differences between Iterator and ParallelIterator. It must be close enough for their needs, but fold is a big example where they're not interchangeable.

Indeed, that's where complexity starts creeping in in bigger apps, and why I think that rayon is a better place to solve this problem - since it has full coverage of the API, and can more easily verify that implementations are interchangeable for parallel and serial versions.

spawn is a particular difficulty. I suppose we might force that to run synchronously, but code may expect independent progress.

Yeah, spawn or, say, manual usage of Mutex and other threading APIs would be particularly tricky, but I think those are rare enough that it could be justifiable to just panic on them in the minimal shim, like std::thread does.

What I'm more interested in is just iterators themselves as that's probably most popular use-case and would allow to easily use same API for parallel and non-parallel builds.

Well, also probably rayon::join since a lot of other things build atop it, but that one also already supports serial execution and shouldn't be an issue.

[RReverser]

See also #591.

Oh yeah, that sounds pretty similar - although the use-cases are quite different, the desired end solution is probably the same.

[arifd]

I'm glad I found this issue, I was surprised I couldn't find out how to do this, so I ended up defining and implementing NoRayon traits with functions that mirror those that I needed and implementing it on the types I needed.

Leaving the code here for future searchers who perhaps don't have the rusty chops to build it for themselves without seeing an example first:

// put this in a file called `no_rayon.rs`, then you can conditionally pull it in as required, i.e.
//
// #[cfg(not(target_arch = "wasm32"))]
// use rayon::prelude::*;
// #[cfg(target_arch = "wasm32")]
// mod no_rayon;
// use no_rayon::prelude::*;

pub mod prelude {
    pub use super::{NoRayonSlice, NoRayonSliceMut};
}

pub trait NoRayonSlice<T> {
    fn par_iter(&self) -> core::slice::Iter<'_, T>;
    fn par_chunks_exact(&self, chunk_size: usize) -> core::slice::ChunksExact<'_, T>;
}
impl<T> NoRayonSlice<T> for [T] {
    fn par_iter(&self) -> core::slice::Iter<'_, T> {
        self.iter();
    }
    fn par_chunks_exact(&self, chunk_size: usize) -> core::slice::ChunksExact<'_, T> {
        self.chunks_exact(chunk_size);
    }
}

pub trait NoRayonSliceMut<T> {
    fn par_chunks_exact_mut(&mut self, chunk_size: usize) -> core::slice::ChunksExactMut<'_, T>;
    fn par_iter_mut(&mut self) -> core::slice::IterMut<'_, T>;
}
impl<T> NoRayonSliceMut<T> for [T] {
    fn par_chunks_exact_mut(&mut self, chunk_size: usize) -> core::slice::ChunksExactMut<'_, T> {
        self.chunks_exact_mut(chunk_size);
    }
    fn par_iter_mut(&mut self) -> core::slice::IterMut<'_, T> {
        self.iter_mut();
    }
}

Also adding these keywords for searchability: fallback, fall back, polyfill, poly fill

[RReverser]

Yeah I hope there will be some further discussion on this, the conversation above just stopped.

[tmpfs]

This would be very useful for us. We have a scenario where we are compiling to webassembly and transient dependencies use rayon so we used wasm-bindgen-rayon (thanks @RReverser ) to get the code to compile.

However this means that LLVM will enable shared memory and thus SharedArrayBuffer is used. The big problem for us is that we want to use the webassembly module in a browser extension where we cannot use SharedArrayBuffer due to the Spectre mitigations.

Being able to have a shim mode that could completely disable threading would solve the problem for us.

[cuviper]

Light sketch:

• An environment variable can override Registry::new to create no threads at all.
  • Or maybe check that in ThreadPoolBuilder::get_num_threads, returning 0 if an explicit count is not set.
• Registry::inject can notice when it has no threads and instead execute the job immediately, blocking.
• Caveat emptor: some code may deadlock if there are no threads to make independent progress.

[kornelski]

I second the request. I need to support single-threaded WASM in my libraries (the experimental WASM threading support is too complex and not supported well enough).

I've been using a shim like this:

https://github.com/ImageOptim/libimagequant/blob/main/src/rayoff.rs#L34
https://github.com/kornelski/dssim/blob/main/dssim-core/src/lieon.rs

but it would be best if it could be supported as a compile-time Cargo feature in rayon itself, so that I can apply this to the whole crate instead of faking APIs one by one.

[RReverser]

(the experimental WASM threading support is too complex and not supported well enough)

@kornelski Offtop:

If by "not supported well enough" you mean browsers, then Wasm threads are actually supported in all major browsers nowadays.

If you mean on the Rust side, yeah, there is no built-in support unfortunately. To cover at least Rayon use-cases and work around browser bugs, I created the wasm-bindgen-rayon, but, yeah, it still has the usual Wasm threads caveats in terms of usage (e.g. you can't use it from the main browser thread).

[RReverser]

An environment variable can override Registry::new to create no threads at all.

@cuviper I'm not aware of cases where we'd need to disable multithreading at runtime tbh. Moreover, for cases like Wasm compile-time flag would be preferable because it would allow to completely remove Rayon threading code & stdlib thread panics code from the resulting Wasm module, leading to smaller network sizes.