Add a light client code that compiles in the browser

[tomaka]

This is a long term issue that I'm opening in order to keep track of PRs moving towards this direction.

We can't directly compile node for wasm32-unknown-unknown. Instead, we should create a new library (named for example substrate-light-wasm) that exposes an API usable from JavaScript, thanks to wasm-bindgen.

Once that crate exists, we can use wasm-pack to generate .js and .wasm files from substrate-light-wasm. The .js can then be used from regular JavaScript code.

Some challenges include:

• We can't use TCP/IP. Libp2p already provides a wasm-ext transport that can be used to "import" a transport from the JavaScript world, such as WebSockets.
• The RPC needs either to be disabled, or be replaced with something that compiles for the browser.
• The telemetry needs either be disabled, or be replaced with something that compiles for the browser. I have a WIP branch that replaces the telemetry with libp2p.
• We need to remove background threads, and have a fall-back for background tasks when tokio isn't available.
• The tokio dependencies need to be replaced with dependencies to their sub-crates: tokio-io, tokio-codec, etc.
• ring and rust-crypto don't compile for WASM. As far as I know, only diffie-hellman exchanges aren't possible through pure-Rust-only crates at the moment. Libp2p uses WebCrypto as a replacement.

[rphmeier]

The tokio dependencies need to be replaced with dependencies to their sub-crates: tokio-io, tokio-codec, etc.

Eep. The facade crate is way nicer.

[tomaka]

Eep. The facade crate is way nicer.

The reason for not using tokio is rust-lang/cargo#4866.
We need to have for example tokio-tcp enabled only on non-WASM platforms, but you can't disable tokio's features per-platform because of this Cargo issue.

[Demi-Marie]

While having a light client that compiles to wasm is nice, I believe that doing anything security-critical in the browser is a bad idea, as a compromised server or an MITM attack could allow compromise of the client’s secrets. Two solutions are:

• Use a browser extension (WebExtension) instead. These run with enhanced privileges, so they can protect themselves, and are signed, so MITM et al are not issues.
• Require that a browser extension is installed that checks the signatures on the actual client.

[gterzian]

@tomaka Has a dedicated web-based client, using web-apis, been considered? It could offer some solutions to the problems mentioned above, such as:

We need to remove background threads, and have a fall-back for background tasks when tokio isn't available.

Use web-workers

The RPC needs either to be disabled, or be replaced with something that compiles for the browser.

You could make a global object available, with methods that could be called from within a browser console(and probably other things like extensions).

ring and rust-crypto don't compile for WASM. As far as I know, only diffie-hellman exchanges aren't possible through pure-Rust-only crates at the moment. Libp2p uses WebCrypto as a replacement

Use available crypto web APIs instead? Is that what libp2p is already doing?

The telemetry needs either be disabled, or be replaced with something that compiles for the browser.

Use the performance-timeline?

Finally, for executing wasm compatible author code, one could use the available web-apis.

Instead of trying to compile a slimmed-down version of Substrate entirely in wasm, and then run it as a wasm module on the web, I think it might be better to build a light-client from scratch in Javascript using all available web-apis.

[gterzian]

@demimarie-parity

a compromised server or an MITM attack could allow compromise of the client’s secrets

One could probably use an extension as you suggest for additional security, and something like Content-security-policy and Strict-Transport-Security and Preloading_Strict_Transport_Security could also address some issues.

Use a browser extension (WebExtension) instead. These run with enhanced privileges, so they can protect themselves, and are signed, so MITM et al are not issues.

Doesn't the signature of an extension need to be checked in some way via communication with a server? If so, that server could also suffer from a MITM attack. In that case using a extension is not more secure than using a normal website with proper use of various security APIs that are available.

I believe that doing anything security-critical in the browser is a bad idea

For the security of usage of Web crypto API's in general, see https://www.w3.org/TR/WebCryptoAPI/#security-developers, for key storage in particular, see https://www.w3.org/TR/WebCryptoAPI/#concepts-key-storage

[tomaka]

Use web-workers

Web workers aren't available from WASM yet.

You could make a global object available, with methods that could be called from within a browser console(and probably other things like extensions).

This issue is about "managing to compile something at all". We obviously want the user to somehow interface with the node, but it's too early to discuss this.

Use available crypto web APIs instead? Is that what libp2p is already doing?

Yes libp2p is doing that.

WebCrypto is however an asynchronous API that's not pluggable everywhere. We were able to use it in libp2p because we use it in an asynchronous handshake. Replacing a straight-forward Signature::verify(...) -> bool method with a method that returns a Future is quite involved.

It is also well-known that WebCrypto in general is an extremely crappy API that is best avoided.

Use the performance-timeline?

The problem of the telemetry is that it uses an HTTP client on top of mio, which isn't available from the browser.

I think it might be better to build a light-client from scratch in Javascript using all available web-apis.

PRs welcome! The point of compiling the existing Substrate code for the browser is to not have to recruit 20 JS devs to do the equivalent JS code, and to relieve teams that build on Substrate from having to recruit 20 JavaScript devs to do the JS equivalent of their chain.

for key storage in particular, see https://www.w3.org/TR/WebCryptoAPI/#concepts-key-storage

The text mentions that, for additional security, the private key isn't extractable. Unless we use WebCrypto everywhere, we do want it to be extractable.

[amaury1093]

I think it might be better to build a light-client from scratch in Javascript using all available web-apis

You're probably aware of this, but just in case: https://github.com/polkadot-js/client aims to do that.

[gterzian]

Web workers aren't available from WASM yet.

Isn't the entirety of Web api's available from wasm code running in the browser?

See: "Although wasm modules themselves can largely only manipulate numbers directly, they can import any arbitrary function which gives wasm full access to the web platform, DOM and all. From day one WebAssembly is all about reusing and enhancing the web platform experience, avoiding the need to reinvent the wheel for new functionality." from https://rustwasm.github.io/2018/10/24/multithreading-rust-and-wasm.html

The point of compiling the existing Substrate code for the browser is to not have to recruit 20 JS devs to do the equivalent JS code, and to relieve teams that build on Substrate from having to recruit 20 JavaScript devs to do the JS equivalent of their chain.

I'm doubtful compiling a significantly working version of Substrate into WASM is feasible, let alone one that can run inside a web-browser environment. The reason for this is that the environment of a web browser would limit the kind of work you could inside the wasm code, unless you were to use web-apis, and only to the extent of the capabilities they would make available(I can't say whether those apis would actually offer enough capabilities for it to work at all).

If the goal is to build a re-usable web-based light-client compatible with Substrate to run in a web browser, I think it's best to focus on exactly that, and I think it's going to have to involve building something with available web-apis in mind from the start.

What resources to allocate to such a project, if any, is another discussion.

You're probably aware of this, but just in case: https://github.com/polkadot-js/client aims to do that.

That sounds like a more appropriate way of achieving that goal, and I assume the reason that wasn't considered initially in this issue is that it wasn't build with other Substrate-based chains in mind?

The problem of the telemetry is that it uses an HTTP client on top of mio, which isn't available from the browser.

A good example of the need to build this with web-apis in mind, if at all, using something like the performance-timeline.

It is also well-known that WebCrypto in general is an extremely crappy API that is best avoided.

Even if we manage to compile something to WASM, you won't have access to system APIs that wouldn't be available to Javascript running in the browser.

If you try to do all the crypto "natively" from the wasm code, you risk ending-up with a user experience where every crypto call will block everything else(including potential user feedback while work is in progress, although maybe web-workers could alleviate that, and that again requires using such web-apis).

Also, unless you use WebCrypto, I don't know how you could do things like truly-random value, since your wasm code is essentially running "on a web-page", similar to equivalent code in JS. WebCrypto is the only way to access system-level crypto capabilities from such an environment, and whether the code is JS or wasm doesn't materially change that fact, as far as I can tell.

The text mentions that, for additional security, the private key isn't extractable. Unless we use WebCrypto everywhere, we do want it to be extractable.

It appears that one can extract the keys if necessary.

"extractable: Reflects the [[extractable]] internal slot, which indicates whether or not the raw keying material may be exported by the application." https://www.w3.org/TR/WebCryptoAPI/#dfn-CryptoKey-extractable

[tomaka]

Isn't the entirety of Web api's available from wasm code running in the browser?

Yes, but what's blocking is that you need to be able to share memory between the workers. JavaScript workers were designed to be totally isolated from one another, and only communicate by passing messages (JS messages, so weakly-typed).

The reason for this is that the environment of a web browser would limit the kind of work you could inside the wasm code, unless you were to use web-apis

The only things we need are networking, timers, and a file system. The file system would be secondary, as we would use a memory database for the initial prototype. Networking and timers are solved in libp2p already, and part of the effort is to propagate this to the rest of the Substrate code base.

If you try to do all the crypto "natively" from the wasm code, you risk ending-up with a user experience where every crypto call will block everything else(including potential user feedback while work is in progress, although maybe web-workers could alleviate that, and that again requires using such web-apis).

While we cannot split the Substrate code between multiple web workers, we can run Substrate as a whole in a single web worker and make the JavaScript UI on top of it run separately. Nothing would be blocking.

Also, unless you use WebCrypto, I don't know how you could do things like truly-random value, since your wasm code is essentially running "on a web-page", similar to equivalent code in JS.

We cannot avoid WebCrypto entirely, but we can avoid using it where we don't strictly have to. The rand crate already runs on WASM-browser and uses WebCrypto.

[rphmeier]

I think it might be better to build a light-client from scratch in Javascript using all available web-apis

You're probably aware of this, but just in case: https://github.com/polkadot-js/client aims to do that.

In general, I think this is a good project but the undertaking of maintaining two separate implementations of:

• pluggable consensus
• warp sync
• GRANDPA light client
• etc; etc;

is going to be super hard, and only reasonable to do once things have settled down in those protocols. So in the interest of saving time and not repeating ourselves, it's probably way easier to plug in a bunch of the Substrate Rust code (which, BTW, we write systems-level stuff in Rust for a reason...) into a WASM-friendly library that wrapper JS can invoke.

Although what I've chatted with @folsen a bunch more about is possibly an even more viable way of getting light clients into browsers: getting browser builds that just include light clients.

[gterzian]

Before answering to some points made above, on a higher-level, my point of view is that it's not feasible to try to compile the current Rust implementation of Substrate, or a subset of it, into wasm so that it can run in a browser. It wouldn't be feasible to try to compile a web-browser to wasm to make it run on the Substrate runtime either.

Why? Because Substrate is like a "browser" for it's own runtime. It's the thing that runs blobs of wasm, making a dedicated API available to those running blobs, just like a Web browser runs a web page and make various APIs available to the JS/HTML/Wasm code running on it.

A subset of the Substrate protocol could very well be implemented to run on the Web, and a subset of a web-page could very well run on the Substrate runtime, and I'm arguing that this would require a targeted implementation of a given subset of a Protocol.

Also, I'm not buying the argument that adding to the Rust implementation of Substrate, or large parts of it, the constraints that it needs to:

1. Complile to wasm.
2. Run on the Web

Is in any way easier than writing an implementation from scratch targeting the web(which I'm not saying should be done right now).

Those additional constraints make it harder to write the Rust implementation of Substrate, and getting a potential wasm subset to run on the web will be hard in itself.

A dedicated implementation of the Substrate protocol, would have the benefit of not influencing the requirements of any other implementation, Rust or otherwise, and to target a specific platform giving it the benefits of being built from scratch with available APIs in mind(Remember those "HTML5 mobile app" that everyone was excited about? They didn't work as well as their equivalent built with native APIs, and it took years and huge layers of glue like React Native for something faintly resembling "HTML5 mobile apps" to work properly).

@tomaka

Yes, but what's blocking is that you need to be able to share memory between the workers. JavaScript workers were designed to be totally isolated from one another, and only communicate by passing messages (JS messages, so weakly-typed).

Could a light-client be implemented using such message-passing?

The only things we need are networking, timers, and a file system. The file system would be secondary, as we would use a memory database for the initial prototype. Networking and timers are solved in libp2p already, and part of the effort is to propagate this to the rest of the Substrate code base.

The web offers you API's for all of these things, and it seems there is no need to "solve" further problems in Substrate, instead one could try doing things from scratch using the web.

While we cannot split the Substrate code between multiple web workers, we can run Substrate as a whole in a single web worker and make the JavaScript UI on top of it run separately. Nothing would be blocking.

That's a decent idea, and I think the "whole of substrate" would not run well inside of a single web-worker. Your gains with such an approach would likely be limited to the ability to show a spinning wheel in the UI that would indeed spin as opposed to block. You could off-course split up the substrate part into various workers, and that would require further work inside substrate to use those APIs(which I don't recommend in the name of keeping things as simple as possible).

The "run it all in a single worker" approach would also require glue code for the JS code to interact with the worker, and for the "substrate" part to communicate "events" to some JS glue layer between the worker and the UI.

In other words, you might end-up having to write an embedding layer in Javascript, which could end-up almost as complicated as writing a light-client in JS from scratch.

We cannot avoid WebCrypto entirely, but we can avoid using it where we don't strictly have to. The rand crate already runs on WASM-browser and uses WebCrypto.

The code inside of rand that is necessary to make that work, is a good example why we should avoid going in that direction. This sort of stuff is manageable inside a single well-focused library, if we start adding these kind of conditionals in Substrate it risks getting too complicated very quickly.

@rphmeier

it's probably way easier to plug in a bunch of the Substrate Rust code (which, BTW, we write systems-level stuff in Rust for a reason...) into a WASM-friendly library that wrapper JS can invoke

My point is that the wasm code running on the web will not be "system-level", your only access to system-level stuff is via the APIs provided by the web. So the Substrate Rust implementation should indeed focus on doing "system-level" stuff, it's hard enough, and not add the constraint that this code needs to compile to wasm itself, let alone run on the web.

So I don't think it would be easier to "plug in a bunch of the Substrate Rust code into a WASM-friendly library that wrapper JS can invoke", I think that approach will make our Rust code look like pre-JQuery browser-sniffing Javascript from 15 years ago.

However I do think parts of Substrate, preferably sequential algorithms not involving any threading or system-level calls, could on a case by case basis be made available in ways that could compile to wasm and used from a potential web-based light-client(or another client running in another wasm compatible environment).

the undertaking of maintaining two separate implementations is going to be super hard

I agree and that's why we shouldn't do it for now, and neither should we make the current Rust implementation more complicated to try and get a free additional implementation.

Conclusion?

I'd say: focus on a Rust implementation that is able to run wasm inside of a given runtime with given APIs available to the wasm code, and forget about compiling "that which makes the runtime run", Substrate, into wasm itself.

A Web implementation of a subset of the Substrate protocol could follow one day, it could offer a runtime for running blobs of wasm too, it's implementation might include various Rust modules compiled into wasm besides other JS modules, yet from my perspective it's not going to be "(a subset of) the Rust implementation of the Protocol compiled into wasm".

[tomaka]

To clear a bit things up, this issue is not "a vague idea that might happen some day". It's more "we know it's possible, we know how to do everything, we know we're not too far away from it, let's just do the appropriate code changes".
These code changes mostly consist in cleaning up the code to make it less reliant on tokio and threads, which also goes in the direction as #2536.

I can see two "invasive changes" (ie. changes that are specifically made for WASM) to make:

• Add a new optional field to NetworkConfiguration so that one can pass some network glue between the Rust code and the browser. Very minimal and probably not controversial.
• Use the wasm-timer crate that provides replacements for std::time::Interval, tokio_timer::Delay and tokio_timer::Interval that also work in the browser. This change would eventually not be needed anymore as the Rust async and wasm teams make progress. While in terms of lines of code the change would be very small, I could understand if it's not wanted, and could just leave that in a separate branch.

[tomaka]

Linking to my old branch, for reference: master...tomaka:wasm-demo
I probably won't bother rebasing this branch anymore. Instead, upstream the changes to master directly.

[tomaka]

My initial prototype was using the informant from substrate_cli. However, in practice, we probably don't need it, as the node would be hidden behind some UI.
Based on this, we don't care about substrate_cli not compiling.

There are therefore only two issues left before the master branch successfully works in the browser:

• All the time-related issues. Instant::now() panics, and Delay/Interval consequently panic as well.
• The slog library (used in the telemetry) tries to use the time crate to get the current time, and that panics as well.
• The substrate_offchain crate doesn't compile because it uses hyper.

[bkchr]

As we will probably never run a validator in a browser, we don't need substrate_offchain to compile. Or?

[tomaka]

As we will probably never run a validator in a browser, we don't need substrate_offchain to compile. Or?

Well, there needs to be some code somewhere that looks like:

#[cfg(not(wasm))]
fn workers() { /* actually use workers */ }
#[cfg(wasm)]
fn workers() { /* nothing */ }

The question is: where? Should that be in substrate-offchain itself, or in substrate-service?

Theoretically it should be possible to make offchain workers work on the browser by using AJAX, so I went for putting this platform-dispatch in substrate-offchain itself.

[tomaka]

Another issue, which I forgot, is here:

substrate/core/finality-grandpa/src/communication/mod.rs

Line 352 in 64e9a77

let mut executor = tokio_executor::DefaultExecutor::current();

The tokio runtime is not available from the browser, and the three expects below will thus panic.

[rphmeier]

As we will probably never run a validator in a browser, we don't need substrate_offchain to compile

We might build in e.g. fisherman functionality to offchain workers. IF you just want to be a consensus-following node, then we don't really need it.

Note that there can be an expectation of node software to build up indices of on-chain data (independent of consensus) which might be automatically done by offchain workers, which you would be missing in that case.

So I have a preference for offchain to work in WASM, if possible

[pepyakin]

Right now, a node compiled into wasm executes wasm runtimes using wasmi compiled to wasm. As one could guess, this is less than ideal. With the recent refactoring though we have an ability to add other execution engines.

So one thing that we can do now is to try to integrate a web assembly engine based on Web API.

[tomaka]

Two more items to fix:

• The transaction pool uses a threads pool now:

  substrate/client/transaction-pool/src/api.rs

  Line 77 in f44873d

  self.pool.spawn_ok(async move {

• The keystore assumes a file system:

  substrate/client/keystore/src/lib.rs

  Line 85 in f44873d

  fs::create_dir_all(&path)?;

[tomaka]

Here's a branch with hacks that make it work: master...tomaka:wasm-demo-3

[tomaka]

I think this can be closed. Project is a success:

• https://github.com/paritytech/substrate/tree/master/bin/node/cli/browser-demo
• https://github.com/paritytech/substrate/tree/master/bin/node/browser-testing