wasmer-wasix requires Tokio runtime now?

[gluax]

Summary

It seems wasmer-wasix requires you to run a tokio runtime when sys-thread is enabled. And if that feature is not enabled, it panics?

There are not any docs that enable this feature require a tokio runtime.

Is there a way to handle this without a tokio runtime? If so, what's an example?

Additional details

Corresponding snippets from the wasmer-wasix library.

let runtime = self.runtime.unwrap_or_else(|| {
            #[cfg(feature = "sys-thread")]
            {
                Arc::new(PluggableRuntime::new(Arc::new(crate::runtime::task_manager::tokio::TokioTaskManager::default())))
            }

            #[cfg(not(feature = "sys-thread"))]
            {
                panic!("this build does not support a default runtime - specify one with WasiEnvBuilder::runtime()");
            }
        });

impl Default for TokioTaskManager {
    fn default() -> Self {
        Self::new(Handle::current())
    }
}

[theduke]

Yes, wasmer-wasix currently by default uses a tokio runtime for many things, including networking, host file IO, timeouts.

This is not required, there are abstractions that allow replacing everything with other/custom async runtimes.

Note that it needs to be an async runtime, because a lot of of the plumbing builds on async. You can also use things like virtual_io::waker::InlineWaker if you want to avoid that, but it might require some deeper understanding of the functionality.

In particular, you will need to customize the Runtime (with the PluggableRuntime or with a custom trait implementation), and provide that to the WasiEnvBuilder (::runtime(), set_runtime()).

You'll at the very least need an implementation of VirtualTaskManager.
If you don't need networking, you can use virtual_net::UnsupportedVirtualNetworking, otherwise you'll also need your own implementation for the VirtualNetworking trait.

For the file system, by default it's all in-memory and should just work.
If you also want host file system access, you'll need a custom implementation of virtual_fs::FileSystem to replace virtual_fs::host_fs::FileSystem.

[theduke]

Also note that some of the traits build on top of tokio traits like AsyncRead/Write, but you just need those trait definitions, nothing else from tokio.
I think there is a separate crate that provides just the traits.

[gluax]

Got it, I was asking because I'm trying to upgrade from wasmer version 3 to 4.

Before the upgrade, I could do something like the following:

wasi_env
  .tasks()
  .block_on(...);

This migrated to:

wasi_env
  .runtime()
  .task_manager()
  .spawn_and_block_on(...):

But it doesn't even make it to this section now since it panics when I finalize my WasiEnv builder since it's not in tokio environment, as it tries to access tokion::runtime::Handle::current(). This panic didn't happen before the upgrade.

let mut wasi_env = WasiEnv::builder(function_name.clone())
      .envs(call_data.envs.clone())
      .args(call_data.args.clone())
      .stdout(Box::new(stdout_tx))
      .stderr(Box::new(stderr_tx))
      .finalize(&mut context.wasm_store)
      .map_err(|_| VmResultStatus::WasiEnvInitializeFailure)?;

So we now need to create a custom runtime or get tokio to work with our current setup?

[gluax]

Not sure if we use the sys-threads feature outside of having it block async tasks for us aha.

[theduke]

What's your current setup and requirements?

[gluax]

Sure! We are spinning up VMs in multiple threads via the actix actor framework, specifically, using a SyncArbiter from that crate. However, this means within those threads, it doesn't have a Tokio runtime setup. Now, our VM still has to call some async methods written on the rust side, which is why we need the block_on calls. But WasiEnv::build fails since it needs to be in async runtime now... I tried creating a tokio runtime and doing a block on running the VM... but then it panics that it's blocked in one place. I also tried using the System actix to block this method, but while that doesn't panic, it does hold infinitely. So, I think I need to convert it to an async message that is then called by that SyncArbiter. It's, unfortunately, a painful refactor on our side.

[theduke]

You can make an existing tokio runtime accessible with https://docs.rs/tokio/latest/tokio/runtime/struct.Handle.html#method.enter .

[theduke]

But you can also construct a new TokioTaskManager from an existing Handle, and use that.

[gluax]

There is no existing tokio runtime since it's spawned in a new thread. Creating one with the thread and using block on results in a deadlock which is interesting, considering the older version used to spawn a tokio runtime for us.