feat: support wasm

[CPunisher]

Related: #4550

By following rolldown/rolldown#467, I have made an ugly MVP to run rspack in browser environment.

Since I find that support for wasm is a feature that involves lots of small changes in current codebase, I'd like to list all the known steps in the following tasklist and divide the process into 3 stages:

1. Compile rspack to wasm target, i.e. we can get .wasm files by napi build. In this stage we mostly add #[cfg(not(target_family = "wasm"))] to distinguish platforms.
2. Run node_binding(wasm) in the browser. After this we can run some basic examples in the browser.
3. Run @rspack/core in the browser. Since rspack interface exposed by node_binding is missing lots of features, such as option nomalization, essential plugins, loader context, which make it hard to directly use, we also need to make this part avaliable. Actually I have not explored this stage in the MVP.

Compile rspack to wasm target

• napi wasi runtime
• mimalloc, stacker, ...
• tokio: async tokio::fs => sync std::fs
• rspack_futures: its dependency async-scoped depends on tokio multi-thread rt.
• lightningcss: this is stuck by Add wasm compilation cfg for Browsers::load_browserslist parcel-bundler/lightningcss#895
• persistence cache: it dependes on inventory, which supports wasm in 0.3. But there are some big breaking changes. We may temporarily disable it.

Run rspack node_binding in the browser

• napi::bindgen_prelude::spawn doesn't execute in wasm: change it to napi::bindgen_prelude::block_on in a new thread.
• napi Buffer is unavailable in browser, see: https://emnapi-docs.vercel.app/reference/list.html#buffer-related
• tracing subscriber: make it easy to debug in the browser
• SharedArrayBuffer is too small to store rspack error message: See https://github.com/napi-rs/napi-rs/blob/23204f19937065bc4edbf7651d66e6c1e2e089d5/wasm-runtime/fs-proxy.cjs#L131C46-L131C50. We may copy the code to grow it or make a pr to napi-rs.
• Some tokio::spawn may not execute at last
• Some fs api is unavailable with current @napi/wasm-runtime. For example, read_dir normally returns a list of entry object but we can't call any function in the object because it is transfered by serialization and deserialization.
• Pass regex error

Run @rspack/core in the browser

• TODO

[netlify]

✅ Deploy Preview for rspack canceled.

Built without sensitive environment variables

Name	Link
🔨 Latest commit	019dc0d
🔍 Latest deploy log	https://app.netlify.com/sites/rspack/deploys/679798c8923ee00008517981

[codspeed-hq]

CodSpeed Performance Report

Merging #9134 will not alter performance

_{Comparing CPunisher:feat/wasm-2 (019dc0d) with main (cc5e3f6)}

🎉 Hooray! codspeed-rust just leveled up to 2.7.2!

A heads-up, this is a breaking change and it might affect your current performance baseline a bit. But here's the exciting part - it's packed with new, cool features and promises improved result stability 🥳!
Curious about what's new? Visit our releases page to delve into all the awesome details about this new version.

Summary

✅ 6 untouched benchmarks

[hardfist]

Run @rspack/core in the browser. Since rspack interface exposed by node_binding is missing lots of features, such as option nomalization, essential plugins, loader context, which make it hard to directly use, we also need to make this part avaliable. Actually I have not explored this stage in the MVP.

If Rust API is good enough, maybe we don't need to ship the js binding parts to browser, I'm not sure whether we need a light weight implementation or a full implementation, @h-a-n-a @chenjiahan what do you think?

[hardfist]

tokio: async tokio::fs => sync std::fs

seems unnecessary cause the ntaivefs is used for native OS, we can just ship a browser_fs for browser side which even don't need std::fs

[CPunisher]

tokio: async tokio::fs => sync std::fs

seems unnecessary cause the ntaivefs is used for native OS, we can just ship a browser_fs for browser side which even don't need std::fs

This is necessary because:

1. tokio fs feature is not compilable in wasm: https://docs.rs/tokio_wasi/latest/tokio/#wasm-support.
2. with the help of napi, std::fs is easily linked to an external file system provided in js side. See: https://github.com/rolldown/rolldown/pull/467/files#diff-603a6b705934598e015625888d735ef43ae35ffc4cb19aba185610fd7c9eb0dfR18

So I just call std::fs in wasm with #[cfg] in the MVP.

[hardfist]

I mean tokio::fs should only be used in NativeFileSystem

rspack/crates/rspack_fs/src/native_fs.rs

Line 19 in 5a6ed31

pub struct NativeFileSystem {

which maybe not necessary for browser, so you could choose either ways

• change tokio::fs to std::fs in NativeFileSystem when target to wasm
• or don't use NativeFileSystem and passing fs instance to compiler in browser

  rspack/crates/rspack_core/src/compiler/mod.rs

  Line 80 in 5a6ed31

  pub input_filesystem: Arc<dyn ReadableFileSystem>,

I'm not sure which way is better

[hardfist]

rspack_futures: its dependency async-scoped depends on tokio multi-thread rt.

not sure rspack_futures can be replaced with with tokio_stream and wether tokio_stream can be compatible with wasm

persistence cache: it dependes on inventory, which supports wasm in 0.3. But there are some big breaking changes. We may temporarily disable it.

I agree persistent cache maybe not needed for browser at all if it's hard to be compatible

[hardfist]

@CPunisher do you plan to finish all tasks in this PR or separated into multi PRs

[CPunisher]

rspack_futures: its dependency async-scoped depends on tokio multi-thread rt.

not sure rspack_futures can be replaced with with tokio_stream and wether tokio_stream can be compatible with wasm

I'm not sure whether rspack_futures is used for performance, since rspack_futures uses async-scoped to create a isolated tokio runtime. In the MVP I just replace all the usages with join_all. See: CPunisher@7123f19#diff-5eb759d814281903aa0355174051f134df9444a46e9a1ce332cad27eed896215R1051.

[CPunisher]

@CPunisher do you plan to finish all tasks in this PR or separated into multi PRs

I'm also considering, but I think both are ok. Because this feature consist of many small changes, it's very easy to separate it. But it seems that incomplete changes are useless. For example, incomplete stage 1 is not compilable and incomplete the whole feature is not avaliable for users. So what's your suggestion?

[hardfist]

that depends on how long would it take, I suggest when you can successfully compile rspack to wasm target(no need to actually run in browser) we can merge it first and introduce wasm target in CI so we wouldn't introduce wasm incompatible crate in the future.

[chenjiahan]

I'm not sure whether we need a light weight implementation or a full implementation

It would be great if a full implementation could be provided, which allows us to run Rsbuild on Stackblitz.