Support Webassembly

[AchalaSB]

Hey,
Please provide wasm support to this library. Networking is one of the important feature to work on Rust+WASM codes.
Not sure is there any tracker for this. Happy to contribute if any.

Thanks

[carllerche]

Sounds very interesting. Could you provide more context around what wasm platform you are targetting? As far as I know, in the browser, there are no TCP sockets. I'm not familiar w/ other wasm platforms.

[AchalaSB]

@carllerche We need all the network protocols like rpc, tcp, websocket and I/O operations

I felt we should first focus on clang compiler as wasm32-unknown-unknown target is closely embedded with Rust. Its a Rust compiler backend for WebAssembly( without emscripten)

If anyone want to use any Http calls its depends on tokio, hyper, websocket libraries. so we need to make sure all the network protocols can be handled in Rust.
To do so the core is in tokio.rs so I came here for the support.

[davidgraeff]

This is about WASI (https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/) I guess?

[DoumanAsh]

@AchalaSB Support of wasm would require someone to investigate wasm capability.
There are many possible variants when building for wasm, and stuff like tcp/udp is definitely not available on browser.

If you're familiar with wasm platforms, it would be good if you could elaborate on availability of IO primitives and event loops there

[carllerche]

Is there a wasm platform today that supports TCP?

[sfackler]

The wasm32-unknown-wasi target will in the future I believe, but it doesn't look like it does currently: https://github.com/rust-lang/rust/blob/master/src/libstd/sys/wasi/net.rs

[quininer]

Tokio has the best sync primitive implementation, it would be great if it could be used on wasm.

[phritz]

tokio has the best sync primitive implementation, it would be great if it could be used on wasm.

Indeed tokio's non-networking primitives would be extremely useful in wasm in the browser: channels, mutexes, (single-threaded) runtime, etc.

[MikailBag]

tokio with feature flags sync and rt successfully compiles for wasm32-unknown-unknown.
I didn't test if it works though.

[Joshhua5]

This is also a dependency for Sentry.io's rust implementation

[Rochet2]

tokio with feature flags sync and rt successfully compiles for wasm32-unknown-unknown.
I didn't test if it works though.

I tried these feature flags with wasmtime+WASI and it did not seem to quite work.
When waiting on a pending future, the system crashes with the following:

thread 'main' panicked at 'condvar wait not supported', library/std/src/sys/wasi/../unsupported/condvar.rs:23:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: failed to run main module `target/wasm32-wasi/debug/example.wasm`

Seems I would need to somehow work around the current pending future wait implementation.
I tried looking into if there is an API for the tokio runtime yielding control (stop execution) and resume later, but I could not see any such API. Is there one?
It would be very useful in single threaded contexts like Wasm where the control of the program may not be in Wasm. Control should periodically be yielded outside of Wasm.

[d0u9]

Tokio heavily depends on the epoll mechanism underlayer which is not supported in wasm runtime right now as I know. The wasm runtime(not in the browser) is sandboxed and is wrapped by a WASI layer. The WASI provides simple libc abilities and has some constriains on accessing host resource.

[carllerche]

Yeah, but the same points hold. In the browser, there isn't much for Tokio to do. Tokio already supports what it can for wasm, but it isn't much (sync, and very basic runtime)

[justinlovinger]

Yeah, but the same points hold. In the browser, there isn't much for Tokio to do. Tokio already supports what it can for wasm, but it isn't much (sync, and very basic runtime)

I got the same condvar wait not supported error in the browser. It seems Tokio does not support a basic runtime in the browser.

[xtuc]

@justinlovinger I had the same error (see discussion #5029). My understanding is that it's not directly caused by WebAssembly and it's probably something in your program. Does the same program work natively?

[justinlovinger]

It compiles natively, but trying to run it natively would not be trivial. Note, my program is not WASI, it is wasm32-unknown-unknown.

[Darksonn]

It sounds like your program reached a situation where it would block forever without making any more progress. When this happens, you get that panic.

[ranile]

You cannot use Tokio runtime in browser. The browser has its own runtime. You can use Promise based futures using wasm-bindgen-futures

[justinlovinger]

It sounds like your program reached a situation where it would block forever without making any more progress. When this happens, you get that panic.

The program ran successfully before tying to use Tokio in place of wasm-bindgen-futures.

You cannot use Tokio runtime in browser. The browser has its own runtime. You can use Promise based futures using wasm-bindgen-futures

It was my understanding this issue is to discuss Tokio support in the browser, I was under the impression this issue was closed, possibly erroneously, due to the belief Tokio supported the browser, hence the statement

In the browser [...]
Tokio already supports what it can for wasm, [a] very basic runtime.

Tokio support in the browser is desirable for my project to better support local testing and future native support.

[rjzak]

Can you submit a PR to address this?

[Darksonn]

Tokio supports what it possibly could support in the browser without using a completely different implementation for the browser. Admittedly, what it supports is rather limited.

[justinlovinger]

What is Tokio expected to support in the browser? As far as I can tell, it does not currently support spawn, spawn_local, or .await.

I guess more accurately, it does not seem to support await, and I know of no way to use spawn or spawn_local without an await to keep a function from completing before spawned tasks.

[Darksonn]

Can you perhaps post the program you tried?

[justinlovinger]

The program is nearly 5000 lines of code, but I found a minimal reproducible example,

use futures::{channel::mpsc::unbounded, StreamExt};
use tokio::task;
use wasm_bindgen::prelude::*;

#[tokio::main(flavor = "current_thread")]
#[wasm_bindgen(start)]
pub async fn main() {
    let (_tx, rx) = unbounded::<i32>();
    task::spawn(rx.for_each(move |_| { async {} })).await.unwrap();
}

rx.next caused the same error. I was not able to reproduce the error without rx.for_each or rx.next, but I cannot say if this is the only code in the full program resulting in a condvar wait not supported error.

[Darksonn]

This program sleeps forever. See my earlier comment:

It sounds like your program reached a situation where it would block forever without making any more progress.

That you get a panic in this situation is documented here.

Note also that if the runtime becomes indefinitely idle, it will panic immediately instead of blocking forever.

I would be happy to accept a PR that improves the error message.

[justinlovinger]

Note, the full program is currently trying to use Tokio like,

task::spawn(optimization_events_fut);
task::spawn(view_fut);
let local = task::LocalSet::new();
local.spawn_local(optimization_fut);
local.spawn_local(controller::run(
    Arc::clone(&data_model),
    Arc::clone(&optimization_model),
    optimization_events_1,
    view_events,
));
local.await;

It is designed like a series of actors, each sending events and receiving commands in response to events. The main timer is implemented with gloo_timers due to a lack of timer support in Wasm without WASI. From the perspective of the whole program, it does not sleep forever. However, I can see how, from the perspective of Tokio, it may appear to sleep forever. Is there no way to overcome this?

Edit: would it be accurate to say "Tokio cannot become idle for any length of time"?

[Darksonn]

Would it be accurate to say "Tokio cannot become idle for any length of time"?

In the browser, yes. Web workers might be able to sleep using tokio::time, but I haven't tried it.

[justinlovinger]

Is there then no desire to see Tokio work in the browser, at least enough for the runtime to be able to pause? This issue was closed in favor of WASI, but as far as I am aware, WASI support will do no good in the browser.

I know libraries exist to generating timing Futures in the browser, but I am not familiar enough with the internal workings of Tokio to know what would be required to integrate these to fix Tokio.

[kaimast]

I would love to have the same runtime for native and web apps, so I don't have to modify my code when compiling for different targets.

Not sure if this is even feasible though with the current state of WebAssembly in the browser?

[ranile]

You can use the prokio crate, which aims to be a pseudo-runtime which uses tokio and JS Promises as needed. It's also is going to be used in Yew: yewstack/yew#2893

I'm not sure if bringing what wasm-bindgen-futures does (wake futures on the microtask queue tick) to Tokio is a good idea, as tokio is it's own standalone runtime and should stay that way

[Darksonn]

It would be very nice to have Tokio work in the browser, but it would require adding a completely separate runtime implementation just for that use-case. Furthermore, our IO types would not work in browser-wasm, so the only new Tokio api this would let you access is timers. It doesn't seem worth it to me.

[justinlovinger]

I was recently informed WASI can theoretically work with wasm-bindgen for the browser,

WASI is just a standard set of imports. Browsers do not implement them by default. But they can be implemented in a browser context, much like wasm bindgen generates its other imports.

https://discord.com/channels/442252698964721669/443151097398296587/1032831725761470567

If this is true, WASI support in wasm-bindgen alongside WASI support in Tokio is probably the most practical solution to Tokio in the browser.