meta: Stabilize WASI support

[carllerche]

This tracks outstanding issues to stabilize WASI support for Tokio.

Initial PR: #4716

Known issues:

• tests were disabled and need to be re-enabled
• tokio::time can be stabilized.
• Use target_family = "wasm" once MSRV becomes 1.54. Current workaround: #4865
• WASI lacks many APIs needed for getting Tokio examples working
  • TcpStream::connect
  • TcpListener::bind

Refs: #1597

[rjzak]

1. Since Support Webassembly #1597 was closed, perhaps track the differences between the three WebAssembly targets supported by Rust:
   • wasm32-unknown-emscripten
   • wasm32-unknown-unknown
   • wasm32-wasi
2. Also, add an integration test which:
   • Compiles a test application in Rust to Wasi which accepts connections
   • Runs the test Wasi application (via Wasmtime library)
   • Connect and ensure it worked
   • Cleanly shutdown the Wasi app
3. Add wasm32-wasi CI test: cargo hack test --each-feature --exclude-features rt-process-signal,rt-multi-thread,rt-net --target=wasm32-wasi (currently has an unused code complaint in the Runtime module for one of the combinations)

[MediosZ]

For missing APIs like TcpStream and TcpListener, I tried to replace std with wasmedge_wasi_socket and it works.
But this requires using WasmEdge and is not compatiable with other runtimes.
Some refs:
mio
tokio

[umgefahren]

Since WASM Threads are already support by Rust Std (although in nightly with build std) would it be possible to bring the multi thread runtime to WASM?

[Darksonn]

I'm sure we will support it eventually, but it is not a priority right now.

[frewsxcv]

Since WASM Threads are already support by Rust Std (although in nightly with build std) would it be possible to bring the multi thread runtime to WASM?

@umgefahren Do you have a link to the PR where this was implemented in std?

[rjzak]

There is some code for Wasm threads, but most of it is empty or unsupported: wasm/atomics/thread.rs.

The same is true for Wasi, wasi/thread.rs.

There has been some progress on Wasi threads, the Wasi-Libc project has some pthread functions https://github.com/WebAssembly/wasi-libc/search?q=pthread, but most seem to be in the musl library. Even if Rust or C had code for Wasi threads, this still requires a Wasi runtime to be able to use it, and I don't think we're there yet.

[milesrichardson]

has anyone tried somehow replacing threads with web workers? intuitively that seems like it would work. I don't know enough about Tokio or Rust to say for certain, but it seems like a natural fit considering that "the Worker interface spawns real OS-level threads".

I searched for "wasm threads web workers" which led to some prior work with emscripten (e.g. https://github.com/boyanio/wasm-workers). Looks interesting. Also gregbuchholz/threads looks promising, claiming successful multi-threading in the wasm32-unknown-emscripten target.

I also found the WebAssembly/threads repository with proposals and discussion around adding threads to WebAssembly. That would be cool, but until then, is there any reason why a runtime like Tokio couldn't provide a Thread implementation for platforms that support the Web Worker API?

Is the idea to basically implement std::Thread for the wasm target (sys/wasm/atomics/Thread) with some platform-specific analog to a "thread" (see the Unix implementation for example)?

This sounds feasible, at least in terms of the features available to Web Workers, like postMessage, transferrable objects, SharedArrayBuffer, Atomics, etc.... right?

[jeffparsons]

FYI: Wasmtime just recently landed thread support: bytecodealliance/wasmtime#5484 (not yet in any release)

[BarbossHack]

"Announcing wasi-threads" (proposal)
https://bytecodealliance.org/articles/wasi-threads

[banocean]

rust-lang/rust#93158 👀

[NobodyXu]

I wonder if WASIX would also be worth supporting?

It looks like they also have their own forked version of tokio for this https://github.com/wasix-org/tokio/tree/wasix-1.24.2

Edit:

They also have a forked libc and a forked mio.

[BarbossHack]

« WASI threads, implementation of wasm32-wasi-preview1-threads target

This PR adds a target proposed in rust-lang/compiler-team#574 by @abrown and implementation of std::thread::spawn for the target wasm32-wasi-preview1-threads »

rust-lang/rust#112922

[jeffparsons]

I wonder if WASIX would also be worth supporting?

See also the open issue about supporting WasmEdge, which seems to be a similar situation: both have their own Tokio fork, but presumably would like to have support upstream in Tokio so they don't need to tell their users to put in an override.

I anticipate that some of the hunger for runtime-vendor-specific extensions from end users of Tokio will fade as WASI Preview 2 comes together later this year, but I definitely don't understand the space well enough to say for sure, or if the vendors themselves would still hope for some upstream support. E.g. if a vendor adds a special IO primitive to their runtime that is faster than what's in WASI by accepting a more relaxed security posture, they might want their users to be able to opt in to using that.

So I guess the question on my mind is one of policy: would the Tokio project want to carry vendor-specific patches? For small things only? For "support WASI Preview 1 long term even after Preview 2 ships"? Yes, but only until WASI itself hits 1.0?

[NobodyXu]

WASI Preview 2 comes together later this year

That will be great, hopefully it will end this vendor-specific extension.
And perhaps wasm should consider supporting vendor-specific extension officially.

[Darksonn]

We already have unstable support for wasi, and I don't mind PRs with updates to our unstable support as long as they aren't too complicated.

[jeffparsons]

That will be great, hopefully it will end this vendor-specific extension.
And perhaps wasm should consider supporting vendor-specific extension officially.

I don't think this is going to go away entirely, and it's not clear to me what additional support in Wasm for vendor extensions would look like beyond what is already afforded by the WebAssembly Component Model.

I suspect there will always be demand for compilers to support vendor-specific extensions, or something equivalent. E.g. Wasmer's WASIX provides POSIX-compatible APIs so that you can compile existing code bases (without invasive modifications) to Wasm, and then run them anywhere. To achieve this today, they have their own rustc fork, and their own tokio fork. And I suspect we're going to see more of that sort of thing from other vendors.

I wonder if this situation could be simplified by the Rust project itself owning a WIT world that defines what rustc needs of a complete OS? Then rustc itself, tokio, etc., could target this world, and then WASIX and other compatibility layers can provide an adapter from this Rust-specific World to their API. The indirection can be compiled away, so I don't think the overhead of an extra layer would be an issue. Then instead of rustc and L different libraries needing to carry support for V different vendor-specific extensions, those extensions would simply implement one world defined by Rust.

Unless/until we ended up with a single standard "full OS" world (presumably) defined under the umbrella of the WebAssembly project you might still have a world defined by each language toolchain that's interested in this sort of thing (e.g. Zig, Nim, etc.) but an adapter component is a lot lighter-weight than managing an entire fork of rustc or Tokio! And it might nudge us in the direction of ending up with a single standard "full OS" WebAssembly world, just like how WASIX is nudging things in that direction. Then instead of C compilers plus L libraries all mapping to V different vendors, you'd eventually be able to have each compiler or library map to one world, and each vendor also mapping to that one world.

Slight tangent: POSIX / "full OS" compatibility might be an anti-goal or at least orthogonal to the goals of a lot of people working in the WebAssembly space (the ambition goes a long way beyond "cross platform bytecode for POSIX"), but there is clearly demand, so it might be best to embrace that. I could even imagine it being a "gateway drug" towards WASI adoption.

[banocean]

I think there are few things that should be changed in tracking issue and taken into consideration

• There is ongoing work on wasm32-wasi-preview2 target (they are still proposals)
  • Proposal: add wasm32-wasi-preview2 target, prioritizing wasi-sockets support WebAssembly/wasi-libc#447
  • Add a new wasm32-wasip2 target rust-lang/rust#119616
  • New Tier-3 target: wasm32-wasi-preview2 rust-lang/compiler-team#694)
• WASI Preview 2 introduced a lot of missing APIs as part of WASI specification
• There might still be APIs that still could be exposed but are blocked by "missing APIs" from the same feature (e.g. on preview 1 you couldn't create a new socket, but you could listen on one created earlier by runtime)
• What about preview 3 plans for stream and feature

[Darksonn]

It's nice that wasi is working towards more socket options, but you say that they are still proposals, so it doesn't sound like there is anything for us to do yet?

[rylev]

I just wanted to drop in as one of the folks working on Wasi Preview 2 through the Bytecode Alliance. Hopefully I can clarify what the state of Wasi is, and what we, in the Bytecode Alliance, hope to accomplish in the medium term.

Preview 2 (perhaps despite the name) is not merely a proposal, but is the official next evolution of Wasi (what has been known until now as Wasi has been retroactively renamed to Preview 1) and has been officially adopted by the Wasi Subgroup which is an official subgroup of the WebAssembly Community Group of the W3C. Preview 2 is meant to hint towards the fact that the evolution will continue towards a Preview 3 before ultimately a 1.0. Even though the Wasi subgroup has chosen to not give this version of Wasi the 1.0 version, Preview 2 is meant as an "island of stability" around which the rest of ecosystem can grow. This means there is a commitment to support Preview 2 well beyond the introduction of Preview 3 and any other versions (on the order of magnitude of years) much like Preview 1 support has not suddenly ended with the introduction of Preview 2.

Beyond this, there is significant interest in the Bytecode Alliance to see adoption of Preview 2 within the community which can help inform changes that will need to happen in future evolutions of Wasi. Members of the Bytecode Alliance have already begun implementing Preview 2 support in Tokio and dependent crates such as mio, and we hope that we'll be able to open pull requests soon and work with the Tokio maintainers on landing these changes. Preview 2 should have the required API support to implement everything Tokio needs for single threaded executors.

It is true that Preview 3 will likely have large impacts on the implementation details on Tokio support for Wasi (due to the introduction of first class async in the Component Model), but of course, there will be significant resources to help move Tokio over to Preview 3 when that times comes (which is probably earliest 1 year from now but may be longer).

[dicej]

FYI, I have created temporary forks of both mio and tokio (and also tokio-postgres, for real-world testing), adding WASI Preview 2 support -- most notably wasi-sockets, which is vastly more capable than Preview 1's meager socket support. See https://github.com/dicej/wasi-sockets-tests for examples and links to the forks.

[dicej]

It's nice that wasi is working towards more socket options, but you say that they are still proposals, so it doesn't sound like there is anything for us to do yet?

They're not just proposals. The wasi-libc implementation is feature-complete and in the process of being upstreamed incrementally. I've created a temporary fork of Rust and added complete wasi-sockets support there, as well as the forks of mio, tokio, and tokio-postgres I mentioned above. Meanwhile, @rylev has been working to add wasm32-wasi-preview2 support to Rust, at which point we'll upstream the wasi-sockets changes to all those repos. In short, the implementation work is more or less done -- it's "just" cleanup, testing, and logistics at this point.

One more fun tidbit: we've updated CPython as well, and got all the relevant asyncio tests passing (i.e. all except a few that spawn threads, fork processes, and/or use signals -- features not yet supported in WASI).

[banocean]

Preview 2 (perhaps despite the name) is not merely a proposal, but is the official next evolution of Wasi

Propasal I was talking about is wasm32-wasi-preview2 compile target 😅, the only thing to implement on tokio side should be changing old wasm/wasi related exclusions

Also all Preview 3 stuff was mentioned as a separate point becouse it's introducing stuff related to handling async

[Darksonn]

Thank you for clarifying. I have opened #6323 to track support for preview 2.

[banocean]

Thank you for clarifying. I have opened #6323 to track support for preview 2.

Can you elaborate what part of WASI support that is not part of WASI Preview 2 support

[Darksonn]

@banocean There are two aspects of this:

• Support WASI without requiring --cfg tokio_unstable.
• Support WASI preview 2. (We currently only support preview 1.)

This issue tracks the first aspect, the other issue tracks the second aspect.

[banocean]

@banocean There are two aspects of this:

* Support WASI without requiring `--cfg tokio_unstable`.

* Support WASI preview 2. (We currently only support preview 1.)

This issue tracks the first aspect, the other issue tracks the second aspect.

Now I'm kinda confused, is there any outline when a feature is put under tokio_unstable feature flag?

[Darksonn]

I don't know what you mean by "outline". Putting things under --cfg tokio_unstable is easy, since they are easily removed if they turn out to be problematic or incorrectly designed. Stabilizing them requires more serious thought.

[banocean]

I don't know what you mean by "outline". Putting things under --cfg tokio_unstable is easy, since they are easily removed if they turn out to be problematic or incorrectly designed. Stabilizing them requires more serious thought.

Oh, I through that wasi support under tokio_unstable feature flag was base on the fact that introducing it normally won't comply with the 6 moths MSRV requirement

[yonas]

Some projects have this as a blocker for supporting WASM.

What are the remaining issues before merge?

[banocean]

Some projects have this as a blocker for supporting WASM.

What are the remaining issues before merge?

It's not gonna break stuff since wasix has forked tokio and all wasix implementations are based on using that crate instead of the official tokio one

[Darksonn]

Supporting wasm in Tokio would require that someone who cares about wasm gets involved with Tokio and does the work to make it happen. To my knowledge, there are no blockers to supporting wasi preview 2 other than someone taking the time to implement whatever is missing. But I'm not going to do it — the wasm world is not something I have any involvement in, and we need someone who does have that knowledge to do it.

(Note that the above talks about things like wasi that look like the platforms we support today. Non-web-worker browser wasm is a different story, and is out of scope for the foreseeable future.)

[dicej]

Now that rust-lang/rust#129638 has been merged, I can start working on mio and tokio PRs to support wasm32-wasip2. I'll try to get a draft PR up for mio this week, at least.

[banocean]

Supporting wasm in Tokio would require that someone who cares about wasm gets involved with Tokio and does the work to make it happen. To my knowledge, there are no blockers to supporting wasi preview 2 other than someone taking the time to implement whatever is missing. But I'm not going to do it — the wasm world is not something I have any involvement in, and we need someone who does have that knowledge to do it.

(Note that the above talks about things like wasi that look like the platforms we support today. Non-web-worker browser wasm is a different story, and is out of scope for the foreseeable future.)

The blocker was rust waiting for wasi-libc preview 2 support, looks like it started moving. Only things that could stand in a way on tokio side is previous wasm compatibility or some system apis not getting covered by wasi preview 2 standard.

[Darksonn]

Ah, I see. I did not know that it was blocked on adding support for things in std.

[dicej]

tokio-rs/mio#1836
#6893