Clarify / define Node.js story

[RReverser]

Apologies if this was discussed before, but I haven't found any relevant references so far (aside from ESM integration) and felt it's worth discussing publicly.

---

While I like wasm-pack, wasm-bindgen and companion bindings libraries, I feel that they focus almost solely on the browser side of things.

At the same time, Node.js story doesn't seem to be currently covered very well (neither in prototypes nor plans), despite it being a popular and an interesting platform for running all sorts of mixed JS+Rust executables without resorting to native modules.

---

First issue is that there exists js-sys for generic JS runtime bindings and there is web-sys with web interfaces for browsers, but I would hope to see something like node-sys for predefined bindings for Node.js built-in modules and objects - process, fs, http just to name a few that would be useful from Rust side.

---

Second issue is that on the actual execution side there is currently no easy way to actually run them from command-like.

With Emscripten you can simply preconfigure node as a target.wasm32-unknown-emscripten.runner system-wide once and then cargo run and friends will just work, but running WASM produced with wasm32-unknown-unknown is currently much more involved and requires:

• Manually defining required bindings to e.g. process.argv to read command-line arguments.
• Adding wasm_bindgen(start) and providing conditionally compiled wrappers to use either argv bindings or std::env::args().
• Compiling WASM manually using cargo (because wasm-pack expects only cdylib as a crate type).
• Finding the produced filename (it might be easy for binaries, but examples and such also include hashes so you need to look this up manually).
• Running wasm-bindgen --nodejs ... on the result.
• Finally, running the result.
• On every iteration, repeat from step 3.

For this one, I would hope to see something like wasm-pack run ... that could be used as a target.wasm32-unknown-unknown.runner and would automate all or most of these things above.

[Pauan]

At the same time, Node.js story doesn't seem to be currently covered very well (neither in prototypes nor plans), despite it being a popular and an interesting platform for running all sorts of mixed JS+Rust executables without resorting to native modules.

I think probably the biggest reason is that on the server you can just use native Rust, no need for wasm at all.

And native Rust is excellent on the server: it has great performance, features, and convenience. Trying to use Node is clunky and slow by comparison.

With native Rust you get to use the same types and algorithms on both the server and client (using serde to send data back and forth). So unlike the situation with JavaScript, you don't really gain anything with Node.

To be clear, I think we should support Node, I just think the above reasons are why it hasn't been as big of a priority as the browser.

Of course I can't speak for the entire RustWasm group, but that's just how I see it.

there is web-sys with web interfaces for browsers, but I would hope to see something like node-sys for predefined bindings for Node.js built-in modules and objects - process, fs, http just to name a few that would be useful from Rust side.

This a lot harder than it seems at first glance.

The web-sys crate is auto-generated from official WebIDL. It isn't written by hand. That has a lot of benefits, the most important being that we don't have to do much work: it automatically keeps up to date with the browser changes.

With Node, we would have to write bindings by hand. That's a lot of work! And maintaining the bindings (to keep up to date with newer Node versions) is also a lot of work.

There's absolutely nothing at all stopping anybody from creating a node-sys crate and putting manual bindings in there. In fact, we encourage it. When I say "anybody", I mean anybody, even you!

For this one, I would hope to see something like wasm-pack run ... that could be used as a target.wasm32-unknown-unknown.runner and would automate all or most of these things above.

That sounds like a good idea to me! Could you fill an issue on the wasm-pack repo?

[RReverser]

With Node, we would have to write bindings by hand.

I don't think this is necessarily true. Node.js TypeScript definitions are pretty precise, and there is already wasm-bindgen-typescript that is intended to generate wasm-bindgen code from TS definitons (as far as I know), so this should be possible to automate as well even without WebIDL.

When I say "anybody", I mean anybody, even you!

Sure, but first and foremost I was hoping to see an official positions and support and not just do it on my own (I already lack behind in tracking issues in OSS projects I'm supposed to maintain).

That sounds like a good idea to me! Could you fill an issue on the wasm-pack repo?

Sure.

[RReverser]

With native Rust you get to use the same types and algorithms on both the server and client (using serde to send data back and forth). So unlike the situation with JavaScript, you don't really gain anything with Node.

This is all true but there are cases where you want to migrate slowly, and using WASM for Rust + JS is one of the nicest options, as well as there are cases where user is limited to just Node.js by your provider, and even (my case) where you want to use a complex library that exists only on npm but still write the rest of your code in Rust.

[fitzgen]

As far as officially-documented positions go, we have:

https://github.com/rustwasm/team#3-join-the-rust-and-webassembly-working-group

We are building a Rust and WebAssembly future where:

• The Web is the primary target.
• Rust plays nice with JavaScript, and can surgically replace a performance-sensitive JavaScript module or function without throwing away the existing code base.

We are not focusing on (but still encourage others to experiment with) scenarios where:

• The Web is not the primary target (e.g. WebAssembly as a universal binary format, or an operating system that runs WebAsssembly processes instead of native code).
• A Web application is written in pure Rust, without integrating with any JavaScript.

The focus is on the Web, but we are happy to help elsewhere when we can and it is easy enough to do so. That roughly translates to supporting (but not focusing on) any environment that involves Rust-generated wasm talking to JS, which is what our toolchain can easily support. That includes node!

It would be cool to have a node-sys! That said, I don't think it is the best use of the WG's time to work on node-sys directly, but we would 100% want to make sure that our toolchain supports whatever it needs.

Agreed that wasm-bindgen-typescript could be a good route for implementing this. The typescript frontend is very out of date and doesn't compile right now. If you wanted to invest in it some more, I'd like to do some design discussion so we can avoid some of its current disadvantages in the future.

[xtuc]

I missed your issue @RReverser, sorry. I opened #204 a lot ago, the bundler team happen to own https://github.com/wasm-tool/node-loader. The nodejs integration seems to be quite related to the bundler one.

[RReverser]

@xtuc Yes, thanks, I saw that issue and that's what I referred to in

aside from ESM integration

in the description (and that's also the reason I sent these PRs to node-loader making it possible to run WASM directly at around same time :) ).

As I clarified later though, I'm interested in other parts of it though, which could be already possible even without (experimental) ESM support in Node.js like running WASM via wasm-pack / wasm-bindgen and exposing native APIs.