The future of the wasi-threads proposal

[loganek]

Hi all,

As there was recently not much work on the proposal, I'd like us to define some next steps to move towards finalizing threading support in WebAssembly.

A brief summary of the current state:

• a number of ambiguities around wasi_thread_spawn host call was clarified
• tests have been added
• tooling support:
  • partial implementation of pthread API in WASI libc
  • a PR to enable threads in Rust: https://github.com/rust-lang/rust/pull/112922/files
• Runtime support:
  • Wasmtime
  • WAMR
  • ... ?
• Open questions:
  • should threads be terminated on exiting from start function: return from _start #21
  • should the memory be exported from the module: should memory be exported as well? #22
  • do we need wasi_thread_exit(): Is wasi_thread_exit needed? #7
  • do we need API for querying a max thread count: Query the maximum available parallelism #31
  • should the proposal be included in wasi_snapshot_preview1: change module name to wasi_snapshot_preview1 #47

I think there are two possible ways we can move forward with the proposal:

1. we keep the proposal in its current stage (phase 1) and start work on a proposal for adding threading to WebAssembly (possibly as an extension to the threads proposal.
2. we continue the work on this proposal - define phase 2 advancement criteria (I think we're close), resolve all the remaining questions, define the release plan.

My personal opinion is that we should progress with wasi-threads proposal and accept the risk it becomes deprecated in the distant future (if ever) by WebAssembly threads - this approach will enable a number of use cases in short/mid-term that are currently blocked by lack of standardization of threading in the WASI ecosystem. There's already been a lot of discussions on open questions, and it feels like some of them only need a summary and updated documentation before closing, and some of them can be even resolved as follow-ups after this proposal.

I'm looking for others' feedback on that before taking any next steps.
cc @sunfishcode @abrown @sbc100 @AlexEne @yamt

[sunfishcode]

I think that wasi-threads has been valuable, both in enabling experimentation and in parallelizing some of the implementation work for threads. The implementations of the memory model support, atomics, wait/notify in engines, as well as mutex/rwlock/etc. in toolchains, are things that are good to have with any threading model.

I also think that, as we said up front when wasi-threads was starting, the instance-per-thread model that wasi-threads is built on has some limitations. It prevents linear memory from being encapsulated, it's complicated if we want to link multiple modules together, and in browsers especially it ties threads to some heavyweight resources. These limitations also turn out to make it incompatible with the component model and WASI Preview 2.

Consequently, I propose we pursue option 1. The full standardization process will take time, but I'm optimistic that we can get to the point where we can start building prototypes and catch up to where the current wasi-threads is now on a much quicker timeframe.

[yamt]

Runtime support:

add toywasm and wasmer.

[yamt]

I also think that, as we said up front when wasi-threads was starting, the instance-per-thread model that wasi-threads is built on has some limitations. It prevents linear memory from being encapsulated, it's complicated if we want to link multiple modules together, and in browsers especially it ties threads to some heavyweight resources. These limitations also turn out to make it incompatible with the component model and WASI Preview 2.

does an alternative proposal which doesn't have those limitations really exist?
i have been told it would be proposed "soon" but have never noticed it happened.

unless someome gives me an url of the alternative proposal, i propose to pursue option 2.

[yamt]

Open questions:

add #13 (lack of defined semantics with multiple linked modules)
maybe i should not have closed it.

[abrown]

I was just talking to @pchickey and he has a good write-up on the "should wasi-threads become a part of preview1" in WebAssembly/WASI#545 — might be worth a read. He highlights how the "instance per thread" paradigm is a problem for incorporating wasi-threads in the component model; let me get back to that later.

I think my answer here to "should we (1) propose core Wasm threads or (2) advance wasi-threads" is "both"! But don't get me wrong: I'm not saying we must progress wasi-threads to phase 2. I propose:

• we continue progressing wasi-threads by dealing with the open questions, continuing to add toolchain support (e.g., Rust), improving the testsuite, etc. to continue momentum and support key use cases for all of us on this thread. We don't need wasi-threads to be phase 2 to do this (right?); but, again, this isn't the issue to die on for me (I'm also fine if it did progress; maybe the wasi-common APIs should move to phase 3?). The key thing I would like to point out is that any actual progress to wasi-threads (the technology, not the WASI phase) not only helps those who want to use threads in WebAssembly now, but makes the future core threads more feasible.
• we continue progressing Wasm core threads — if you're interested, please take a look at the draft I've created here. That's going to take longer to become a reality because of various viewpoints held by different parts of the ecosystem. That is fine — if messy — and hopefully your input can help push that effort along. In my mind, due to the similarities, we could use 90% (?) of the progress made in wasi-threads, especially in toolchains, toward making core Wasm threads something we can experiment, and I feel there is real value in making a prototype of core Wasm threads available to us, the users.

Back to the "instance per thread" problem: if the component model and wasi-threads are incompatible, is work on wasi-threads wasted? I would propose that no, I think there is this intermediate period where wasi-threads is still very valuable. Someone may ask, "what if I want WASI but not the component model?" — I can't answer that and it's probably an issue for a different repository! I am going to continue working on making threads available in a portable way in WebAssembly, focusing mainly on the core threads work but helping out as much I can with wasi-threads.

[loganek]

Thanks all for sharing your view on that topic so far. As of writing the post I only knew there are plans to work on Wasm core threads, but didn't know there's already a work in progress started recently by @abrown.

Having threading support in core Wasm sounds reasonable and it's probably good for the long term. However, as pointed out by @abrown, it's going to take time to get there. Until then, I think WASI should provide a standardized solution that satisfies at least a (potentially significant) subset of use cases. If that doesn't happen, I think the adoption of WASI across different teams/companies will be slower as some of the innovations will be blocked, or worst case - might lead to fragmentation and various forks of the spec implemented by different runtimes (which I think we should avoid if we want to compile once, run anywhere).

Given the amount of work and significant progress made as part of wasi-threads, I think the proposal already serves the purpose of being the middle ground between having no threads and the long-term solution. This proposal doesn't fit into the component model and has some limitations, but it does solve a lot of the business problems already. I know there are teams, including my own at Prime Video, who need a stable multi-threading solution in WASI soon. Waiting for Wasm core threads is not really an option, so my team is fully committed to driving wasi-threads proposal forward to the next phases (I think the proposal is nearly ready to advance to phase 2).

If the proposal stays where it is today (phase 1), it will always be considered experimental and its support may be limited by some of the toolchains (an example of that is a discussion rust-lang/rust#112922 regarding tiering of wasi-threads target).

I think we should consider either backporting wasi-threads to preview 1 or potentially relax the requirement for preview 2 and accept exceptions from the rule that everything must fit into a component model. I might be wrong about that, but I expect that it will take some time for a lot of runtimes to support the component model, so including wasi-threads in both preview 1 and preview 2 seems like an option too. I understand the final decision (whatever it is) will not only impact this proposal but the WASI as a whole, so I added the agenda item to the next WASI meeting to discuss it with a broader community (WebAssembly/meetings#1327).

[penzn]

My personal opinion (I am not a very active WASI contributor, so take that as you will) is that standardizing this in WASI would help develop user base, which in turn would help drive adoption upstream. Relying on upstream standardization alone would make the curve of getting it in steeper, by removing possible intermediate steps.

In addition to what thread that @yamt opened shows, there is are a couple aspects of how 'core threads' are going to fit within larger Web standards:

• Wasm instances spawning threads means observable differences for upstream specification, which need to be well understood and corresponding community convinced ;)
• A secondary issue is that current Web approach is less 'pthreads' and more CSP, though if the main point is to use shared memory it might not be that big of an issue

[loganek]

Thank you very much for your valuable feedback. We have engaged in extensive discussions on this matter, involving participants from the WASI subgroup meeting (refer to the agenda item). After careful consideration, the community has reached the following conclusions and agreements:

1. The WASI threads proposal will not be included into any WASI releases. This means it will not be retroactively added to WASI snapshot preview1, nor will it be included in subsequent releases such as the upcoming WASI preview2.
2. The proposal will be finalized, addressing any remaining questions to ensure clarity and to eliminate ambiguities. While WASI threads come with their own set of limitations, they cater to several existing use cases, prompting the need for an immediate threading solution for certain teams.
3. WASI threads will be supported within WASI libc for an extended period. Our team at Amazon Prime Video commits to sustaining this support even after the eventual finalization of the new proposal. If the effort required to uphold WASI threads in the WASI libc becomes substantial (the determination of which is subjective and depends on the challenges posed by maintaining this support), we intend to develop a translation tool. This tool will convert binaries compiled using the new proposal into formats compatible with WASI threads.
4. After addressing all uncertainties, this repository will enter a state of freeze. However, the README file will remain updated with resources and information pertaining to toolchain developments.

With respect to point 3, you might wonder why we don't simply transition to the new proposal once it's available and supported by both toolchains and runtimes. There are scenarios where runtime updates are not possible for an extended timeframe (beyond 5 years), leaving only the option of updating the WASM binary. For these scenarios, we want to provide the opportunity to use WASI threads today, while also taking advantage of non-threading-related toolchain updates in the future.

I am resolving this ticket now. Please don't hesitate to reopen it if there are further discussions or questions.

Furthermore, if you're interested in helping with the remaining work on WASI threads or maintaining the toolchain support, feel free to reach out to me directly.

[whitequark]

• do we need wasi_thread_exit(): Is wasi_thread_exit needed? #7

I wrote a comment (#7 (comment)) justifying the existence of wasi_thread_exit.