WASI support

[daxpedda]

This issue will be used to collect discussions and issues about WASI support in wasm-bindgen.

Generally speaking wasm-bindgen doesn't currently support WASI, but we should figure out if we want to support it (#3324) or disable it (#3233).

The original plan, outlined here: #1841, now doesn't have the necessary developer support. But the motivation for it, the component model (previously interface types), isn't here yet either.

Input from stakeholders would help a lot!

[csnover]

A library I was working with compiled and linked to a C library using the cc crate, and I believe that wasm-bindgen is not usable in that case unless it supports WASI. (I say ‘believe’ because I was never able to find a channel that was still active to get confirmation from someone authoritative, but at least it seemed like WASI was the ‘blessed’ way to make this work. I at least was unable to find some other way to make it work in the time I had to investigate.)

[daxpedda]

I guess this is one use-case: having WASI dependencies.

But I would argue that this is really a bad workaround, C libraries can target Wasm without WASI or Emscripten just fine. If any WASI function is used it wouldn't work in the browser anyway without a compatibility shim.

On the other hand I understand that the only Wasm target C/C++ libraries usually support is Emscripten, or WASI if they are more modern. As hacky as I described this above, it's a workaround many users could require.

Though I've honestly no clue if you can use a WASI dependency successfully or not with wasm-bindgen, so more input would be appreciated. Thanks @csnover.

[csnover]

Getting a WASI provider into the browser that covered the required system calls was trivial with https://github.com/bjorn3/browser_wasi_shim and required less than 10 LOC change to the bootstrap JS to make it send the required imports when instantiating the module, so this is no barrier.

The third-party C dependency has no knowledge of wasm and used system calls which are available in the WASI shim; I do not understand the comment about it being a ‘bad workaround’ so would be interested to understand more where you are coming from in this regard.

I am not paying attention to the wasm ecosystem to understand what is what, but as an outsider exploring the space for the first time, it looked like WASI being positioned as the standard ABI. Maybe that is just an wrong impression though?

[daxpedda]

Getting a WASI provider into the browser that covered the required system calls was trivial with https://github.com/bjorn3/browser_wasi_shim and required less than 10 LOC change to the bootstrap JS to make it send the required imports when instantiating the module, so this is no barrier.

I agree, it's not a barrier.

The third-party C dependency has no knowledge of wasm and used system calls which are available in the WASI shim; I do not understand the comment about it being a ‘bad workaround’ so would be interested to understand more where you are coming from in this regard.

So I'm searching for the "how it's supposed to be done". WASI isn't supported by the browser, that's why a shim is required.

The "proper" way to do it is to compile to Wasm, not to WASI, which C/C++ is able to. Any C/C++ library not using any system calls will support the Wasm target (barring any complications with build systems 🤣), but as mentioned above, C/C++ libraries that do use system calls, don't commonly support targeting Wasm, but Emscripten.

(I'm using the term "Wasm" here as in the bare target, without WASI or Emscripten.)

---

I'm not against supporting WASI in wasm-bindgen by the way. Just trying to determine how useful, necessary and burdensome it will be.

The original plan for wasm-bindgen was to support WASI anyway, but in a very different way we are now not able to do anymore, see #1841.

[csnover]

I understand what you were saying now, thank you for sharing.

For what it is worth I don’t have much skin in the game at all here; I was just told by someone recently that the emscripten target for Rust did not work, took that claim at face value, saw that WASI was being developed under the WebAssembly group, and assumed this meant the ecosystem had agreed to move toward WASI as the universal ABI for runtimes that need system interface.

I was able to hack wasm-bindgen into working with the wasi target within a day so it seems like it’s mostly there, but it is possible that I didn’t exercise the interface enough since I was only doing it as a proof of concept.

Today I read some discussions in the WASI WG repo and have much more uncertainty as to whether there is actually a commitment to ensure first-class web platform support. Hopefully they are able to resolve that one way or the other since this is the sort of ambiguity that kills standards.

Let me know if there is any other feedback I can provide. Thanks again!

[itsezc]

Not at all sure if its entirely related but as someone who actively uses the web-sys crate, I'd love to see component model come to life (sooner than later) and hope that application developers can get away with little to no refactoring (just a dream); I'd love to see WASI supported.

[gbj]

As the one who proposed #3233 and as the maintainer of a now-large-ish Rust/WASM web framework: I am fine with any solution that is an actual solution.

At present, wasm-bindgen generates bindings that panic if called outside the browser. This is completely fine for me: it means that we can use types like web_sys::Event (and other things that depend on wasm-bindgen) to define event handlers etc. as long as they aren't actually called on the server, in which case they panic. Any framework is perfectly capable of handling this and simply not calling code that will panic in this way.

However, in current wasm32-wasi runtimes server-side compiling with a wasm-bindgen tends to lead to a runtime panic due to unknown exports even if they are never called. This means that wasm32-wasi behaves quite differently from any non-WASM server target.

For additional context see the following

leptos-rs/leptos#295

bytecodealliance/wasmtime#3937

bytecodealliance/wasmtime#5557

[temeddix]

A library I was working with compiled and linked to a C library using the cc crate, and I believe that wasm-bindgen is not usable in that case unless it supports WASI. (I say ‘believe’ because I was never able to find a channel that was still active to get confirmation from someone authoritative, but at least it seemed like WASI was the ‘blessed’ way to make this work. I at least was unable to find some other way to make it work in the time I had to investigate.)

As @csnover mentioned, wasm32-unknown-unknown has too much limitations that it won't work well on web browsers. Major wasm runtimes and major crates including tokio now support WASI, but it's only wasm-bindgen that does not. If we get to support WASI, so many crates that depend on system time and threads will be able to work on web browsers, with the power of WASI polyfills like wasmer-js, and that would indeed be wonderful.

Since wasm-bindgen is the de-facto standard of Rust ecosystem for wasm, IMHO extending support to wasm32-wasi would benefit so many users out there working for the web.

[temeddix]

#3454 seems to be the initial progress toward WASI goal, so I'll leave the link here.

• Align with the "C" ABI #3454

[daxpedda]

As @csnover mentioned, wasm32-unknown-unknown has too much limitations that it won't work well on web browsers.

wasm32-unknown-unknown is not a browser target, which is why it is so limited, but wasm32-wasi isn't either. Hopefully, at some point in the future, a new wasm32-web target can be introduced. This would require the component model though.

Major wasm runtimes and major crates including tokio now support WASI, but it's only wasm-bindgen that does not.

I think in general Rust projects can add support for wasm32-unknown-unknown. Tokio for example, wouldn't work in the browser, AFAIU, even if wasm-bindgen supports wasm32-wasi, see tokio-rs/tokio#5418.

---

I generally don't think that WASI is entirely out of scope for wasm-bindgen. But supporting WASI just to work around issues of getting Rust crates working in the browser, is imo a mistake that will only lead to confusion. wasm32-wasi is not a browser target and probably never will be.

Adding the WASI shim though is something that makes sense to me, because unlike Rust, C/C++ support for the browser is not "really" a thing (except through Emscripten). Rust libraries can use wasm-bindgen, C/C++ can not.

So the way forward here to me is:

1. Add browser support to Rust libraries that need it by using wasm-bindgen for the wasm32-unknown-unknown target. This has worked wonderfully until now, see Winit or Wgpu for example.
2. Find a Rust library that cannot support wasm32-unknown-unknown but can support wasm32-wasi so we can discuss why wasm-bindgen actually needs to support wasm32-wasi. E.g. Tokio is not one of those, it's WASI implementation doesn't work in the browser.

It is obviously very convenient to just slap WASI support onto wasm-bindgen and call it a day, but the point I'm trying to make here is that this is a tradeoff, there are downsides to this. E.g. using Tokio in the browser only to find out that it either will just crash, because it blocks, or if running in a web worker, just never yield back to the browser runtime, making e.g. events never trigger.

I hope this clarifies why I believe that the "wasm32-unknown-unknown target is too limited" argument doesn't make a whole lot of sense to me.

[temeddix]

wasm32-wasi is not a browser target and probably never will be.

Agreed, I was just trying to point out that WASI can run on the browser via polyfill libraries. The main benefit would be std almost fully working just like native, though specific behavior on the web will depend on polyfill libraries(Such as showing the folder dialog to user to access user folder with JS FileSystemDirectoryHandle).

I think in general Rust projects can add support for wasm32-unknown-unknown.

Yeah, this is how the Rust web ecosystem is evolving right now. However, many crates won't work unless each of them implements a solution for wasm that bypasses std by interacting with JavaScript. Simply being able to depend on std would be a huge benefit.

E.g. using Tokio in the browser only to find out that it either will just crash, because it blocks, or if running in a web worker, just never yield back to the browser runtime, making e.g. events never trigger.

Yes, blocking the browser's main JS runtime should never happen. But since atomics are already available in wasm, access to system time and threads via std will possibly make tokio on the web work again. I was looking forward to creating a PR at tokio repo that yields periodically to the JS runtime(Pehaps every 1ms, or even less) once WASI can run in browsers. We can already use wasm_bindgen_futures to spawn Futures in JS runtime, but this doesn't allow us to use extensive tokio features such as task_local!. While it's true that tokio's current WASI implementation doesn't work on the web, a simple PR that makes the runtime yield periodically can do the trick.

Find a Rust library that cannot support wasm32-unknown-unknown but can support wasm32-wasi so we can discuss why wasm-bindgen actually needs to support wasm32-wasi.

Yes, this would be a nice research to make. Thanks for the idea.

[daxpedda]

While it's true that tokio's current WASI implementation doesn't work on the web, a simple PR that makes the runtime yield periodically can do the trick.

Yes, but this can't be added to wasm32-wasi, this should be added as wasm32-unknown-unknown support to Tokio.

I was looking forward to creating a PR at tokio repo that yields periodically to the JS runtime(Pehaps every 1ms, or even less) once WASI can run in browsers.

I would argue that this is pretty crude and probably won't be accepted by Tokio. To properly add support for wasm32-unknown-unknown, I think a completely new executor would be required.

See https://github.com/WICG/scheduling-apis/blob/main/explainers/yield-and-continuation.md for proper yield functionality in JS, but even that I believe is not the way to move forward in Tokio.

[temeddix]

There was an opinion at the forum about this WASI support and ABI mismatch.

• https://users.rust-lang.org/t/fixing-rusts-webassembly-targets/88947

[CryZe]

I don't see an inherent reason why that function couldn't exist on wasm32-unknown-unknown.

[vmx]

I don't see an inherent reason why that function couldn't exist on wasm32-unknown-unknown.

I would expect it to be similar problem as for the WebAssembly support in the getrandom crate.

[CryZe]

That's not really a "problem" as much as it's the "state of the art". The entire ecosystem works that way. At least until there's a separate -web target.

[temeddix]

To me, that looks like a 'problem' because unless a crate aims to support webassembly it doesn't work on the web at all. They have to write additional code or add additional feature to make it work on the web.

With wasm32-wasi, this code duplication is not needed, mainly because it can use std and the standard C ABI.

[daxpedda]

I do disagree with the arguments that adding WASI support directly is a bad idea.

I will try to elaborate on my points later on.

Please do, this is what we need to do here!

For example, they would have to write this...

Actually you can just write:

use web_time::Instant;

fn main() {
    let now = Instant::now();
}

... which works correctly on all platforms.

---

@daxpedda an example is https://github.com/rustpq/pqcrypto. It uses a C library that needs randomness. With wasm32-wasi it's calling arc4random_buf(), which isn't available on wasm32-unknown-unknown.

This is the kind of stuff we do want to support. A linked library can be compiled to WASI to support C/C++ dependencies which can't use wasm-bindgen, but Rust can stay wasm32-unknown-unknown. Though I don't exactly know what arc4random_buf() is, but I assume it relies on WASI random? If it's not something supported by WASI, then I'm not sure what the point here is.

---

To me, that looks like a 'problem' because unless a crate aims to support webassembly it doesn't work on the web at all. They have to write additional code or add additional feature to make it work on the web.

With wasm32-wasi, this code duplication is not needed, mainly because it can use std and the standard C ABI.

I believe this is the crux of the problem here and I'm assuming where you were coming from this whole time.

This is what we ought to discuss here. As far as I've experienced it, slapping WASI support on top will cause issues exactly for this reason.

E.g. Tokio now has WASI support, but doesn't work in the browser. If Tokio wants to add browser support, they can't add it to wasm32-wasi, because that would break WASI support, they have to use a different target, e.g. wasm32-unknown-unknown. So if some of the ecosystem supports the browser through wasm32-unknown-unknown and some of the ecosystem through wasm32-wasi, we will have target incompatibility.

@temeddix I encourage you to get into the specifics here. Which crates did you have problems with? What part of the ecosystem lacks browser support that you would like to see supported? How exactly could WASI support be applied to those crates to add browser support without creating target incompatibility?

[temeddix]

I will organize the list of crates that has issues on wasm32-unknown-unknown soon, after I have my daily tasks done.

Also, I get the point that wasm32-wasi is not for browsers. However, since wasm32-unknown-unknown is also not for browsers, what do you think about adding wasm32-wasi-browser or wasm32-wasi-js target to Rust, which is basically WASI but intended to run in browser JavaScript environment?

Some people do suggest deprecating wasm32-unknown-unknown:

• https://users.rust-lang.org/t/help-us-make-wasm-bindgen-support-more-target/101718

[daxpedda]

Also, I get the point that wasm32-wasi is not for browsers. However, since wasm32-unknown-unknown is also not for browsers, [..]

This is true! It shows too: this is why we are having all these problems here!

[..], what do you think about adding wasm32-wasi-browser or wasm32-wasi-js target to Rust, which is basically WASI but runs in browser Javascript environment?

I'm not a Rust maintainer/member, so you are asking the wrong person. But I doubt anything like that will be accepted by Rust.

The reason why wasm32-unknown-unknown is being used as a browser target is because we don't have a better choice right now, it's only an ad-hoc solution to a problem we can't solve any other way. But I'm not sure how exactly you are going to convince the Rust team that a wasm32-wasi-web target has a purpose if it's exactly the same as wasm32-wasi (or how exactly would it be different? after all JS support can't be added to rustc, unfortunately). Target incompatibility is "just" a downstream issue for rustc as long as an alternative exists: wasm32-unknown-unknown.

I'm not aware of any effort to standardize WASI in browsers, but if there would be, I think that might be an argument. But again, it has to differ somehow to wasm32-wasi.

But, to re-iterate: this is something you have to take up with the Rust team.

[daxpedda]

Some people do suggest deprecating wasm32-unknown-unknown:

• https://users.rust-lang.org/t/help-us-make-wasm-bindgen-support-more-target/101718

I doubt wasm32-unknown-unknown will ever be deprecated. But it is true that it has been misused so far as a browser target, which is very unfortunate.

wasm32-web will happen (:crossed_fingers:), but it requires the component model, I'm not aware of any other way to access browser APIs in rustc without it.

[temeddix]

If wasm32-wasi-js is accepted by Rust maintainers, would it be possible that wasm-bindgen extends support to that target?

Also, I'm wondering even if WASI component model is accepted someday, wouldn't wasm32-wasi-js still be there to allow using that component model with wasm-bindgen in JavaScript environment?

Lastly, maybe wasm32-wasi-js could be a clearer target description for all deno/nodejs/browser. What do you think? Because it's basically Rust in JavaScript environment. Or should it be wasm32-wasi-web?

If I have some consensus from wasm-bindgen side, then at least I can take this and suggest the new target at Rust side, though not sure if it will be accepted.

[daxpedda]

If wasm32-wasi-js is accepted by Rust maintainers, would it be possible that wasm-bindgen extends support to that target?

If it does exist, I don't see why we shouldn't support it in wasm-bindgen, wish I had more input though.

Also, I'm wondering even if WASI component model is accepted someday, wouldn't wasm32-wasi-js still be there to allow using that component model with wasm-bindgen in JavaScript environment?

I'm not aware of any WASI component model, the component model proposal is a Wasm proposal, not a WASI one. So I would say no. I also wouldn't know why anybody would still continue to use WASI in the browser when the component model is available and presents itself as a working alternative.

Lastly, maybe wasm32-wasi-js could be a clearer target description for all deno/nodejs/browser. What do you think? Because it's basically Rust in JavaScript environment. Or should it be wasm32-wasi-web?

I would vote for wasm32-wasi-web.

If I have some consensus from wasm-bindgen side, then at least I can take this and suggest the new target at Rust side, though not sure if it will be accepted.

Personally I would like to know first what this target is supposed to be exactly. But then sure, we can try to get some consensus here.

FWIW, my assumption here is that Rust doesn't (and imo shouldn't) care about wasm-bindgens role in all this anyway.

[vmx]

This is the kind of stuff we do want to support. A linked library can be compiled to WASI to support C/C++ dependencies which can't use wasm-bindgen, but Rust can stay wasm32-unknown-unknown.

@daxpedda Can I today compile a Rust crate as wasm32-wasi and link that with wasm32-unknown-unknown? If yes, pointers would be welcome.

Though I don't exactly know what arc4random_buf() is, but I assume it relies on WASI random? If it's not something supported by WASI, then I'm not sure what the point here is.

arc4random_buf() is just a call to a random number generator which is supported by WASI.

[daxpedda]

This is the kind of stuff we do want to support. A linked library can be compiled to WASI to support C/C++ dependencies which can't use wasm-bindgen, but Rust can stay wasm32-unknown-unknown.

@daxpedda Can I today compile a Rust crate as wasm32-wasi and link that with wasm32-unknown-unknown? If yes, pointers would be welcome.

Yes and no. It should be theoretically possible, though I wouldn't be able to give you any pointers on it as I've never tried it myself, but practically you will very likely hit #3454.

[ranile]

I don't think it's wasm-bindgen's job to provide a shim for WASI. WASI is way more than what could be execute in browser. I still think fixing wasm32-unknown-unknown is the better way to go. Adding yet another wasm target is only going to complicate things, I believe.

I created #3674 to make releasing easier so we can get a release out of wasm-bindgen that supports C ABI

[temeddix]

IMHO, if a shim for WASI is provided for the browser, that would be done on wasm-pack, not wasm-bindgen.

This is being discussed on that side:

• Support for wasm32-wasi target? wasm-pack#654

[temeddix]

I'm not aware of any WASI component model, the component model proposal is a Wasm proposal, not a WASI one. So I would say no. I also wouldn't know why anybody would still continue to use WASI in the browser when the component model is available and presents itself as a working alternative.

I also want to point out that the component model standard states that it's currently being incrementally developed and stabilized as part of WASI Preview 2

• https://github.com/WebAssembly/component-model#milestones

[zhusjfaker]

Sorry, I want to give my 3 points.

1. how wasm-pack would like to promote, I think it's necessary to get more users to access it. But if it only supports the "wasm32-unknow-unknow" browser standard, I think the probable result will conflict with the purpose.

2. Essentially, I see the wasm-pack as taking care of linking to js content for me beyond "cargo build ---target wasm". So I think it would be a good idea to add an experimental flag to the wasm-pack to give the user a chance to select "wasi".

3. as a promotion, most of the users are dealing with normal business. "tokio" and "mysql_wasi" especially the latter can only be compiled under wasm32_wasi at the moment. This makes it painful for users to use "cargo build" instead of "wasm-pack build" for business development.

Based on these three points, I hope that "--target wasm32-wasi" will be supported soon. thank u master , i hope you enjoyed everyday

[temeddix]

Totally agreed, especially on the point that many crates need wasm32-wasi to work properly.

Though the current status of wasm-bindgen is that it cannot support wasm32-wasi directly.

[daxpedda]

"tokio" and "mysql_wasi" especially the latter can only be compiled under wasm32_wasi at the moment.

@zhusjfaker could you elaborate? AFAIK neither would work in the browser. How exactly would wasm32-wasi support in wasm-bindgen help you? What is it you are hoping to achieve exactly?

[zhusjfaker]

This scene is really special, because in addition to the browser, in China there are a lot of NodeJs as a full-stack development, but the complex Crud scene, there are a lot of not only write Node-js using mysql there are also some people are using Rust c++ and other languages together to complete, and even in China there are a lot of cross-end development, try to add in the cell phone, such as iOS platforms The wasm3 project tries to use wasm's package volume to replace JS. If you only consider the browser environment, the limitations may be relatively small.

[daxpedda]

I wasn't aware that NodeJS supports WASI natively, though I'm seeing that it is only experimental.
AFAIK NodeJS should be in scope for wasm-bindgen, but I personally have no clue or connection to NodeJS at all, other maintainers have to chime in on this.

It seems to me that this would be good justification to enable WASI support in wasm-bindgen. This would entail two things:

• Allow to compile to it.
• Make sure it's possible to insert a shim nicely.

I still think it's out of scope for wasm-bindgen to supply it's own shim, which has been said by other maintainers before already.

[zhusjfaker]

i got it thank you

[pfernandom]

Sorry to revive a kind-of old thread, but I was wondering if this is a topic still in discussion.

I've spoken with multiple developers who have expressed interest in integrating WASM/WASI into their server-side NodeJS applications to hand-off memory intensive tasks to other languages like Rust, while still keeping most of the development in NodeJS.

For instance, I have a use case where I'm running an Express server that could use some integration with WASI for handling files directly on the file system (a use case that is possible with WASI but not with WASM); most of the developers in my team are familiar with NodeJS but not with Rust, but they would be open to have a single, critical module being written in Rust.

Being able to use wasm-bindgen to glue both NodeJS and Rust would unlock a lot of options for backend-developers.

The original plan in #1841 still makes sense to me (though I think a good first step would be to work towards supporting WASI with JavaScript, as the. scope is not as wide as in the original plan). If more help is needed to make it happen, I'm open to collaborate on it.

[bjorn3]

You may be interested in jco, which is a polyfill for the wasm component model implemented in javascript. The wasm component model is used by wasip2. It has a shim for wasip1 support too.

[daxpedda]

The original plan in #1841 still makes sense to me (though I think a good first step would be to work towards supporting WASI with JavaScript, as the. scope is not as wide as in the original plan). If more help is needed to make it happen, I'm open to collaborate on it.

The original plan will not come into fruition without additional maintainers, its too big of a change for the resources available right now.

[sneurlax]

I'd like this for tokio, mostly. Otherwise I'll have to use another approach

I am available to contribute work with direction.

[daxpedda]

I'd like this for tokio, mostly. Otherwise I'll have to use another approach

I am available to contribute work with direction.

I outlined above why I believe using WASI for Web support in Tokio won't work as expected.

Otherwise nothing changed really, see #3421 (comment).
I think starting with a proposal how this can be achieved exactly and the motivation behind it would be a good first step.