Build is broken, trying to fix it

[superboum]

Build is broken on main:

$ git clone git@github.com:Ekleog/kannader.git
Clonage dans 'kannader'...
remote: Enumerating objects: 67062, done.
remote: Counting objects: 100% (67058/67058), done.
remote: Compressing objects: 100% (7993/7993), done.
remote: Total 67062 (delta 59575), reused 65030 (delta 57806), pack-reused 4
Réception d'objets: 100% (67062/67062), 49.57 Mio | 6.31 Mio/s, fait.
Résolution des deltas: 100% (59576/59576), fait.
$ cd kannader/
$ nix-shell
$ cargo --version
cargo 1.61.0-nightly (65c8266 2022-03-09)
$ rustc --version
rustc 1.61.0-nightly (285fa7ecd 2022-03-14)
$ cargo build
error: package `arbitrary v1.1.7` cannot be built because it requires rustc 1.63.0 or newer, while the currently active rustc version is 1.61.0-nightly

I bumped your rust toolchain to the latest nightly:

$ cargo --version
cargo 1.78.0-nightly (7bb7b5395 2024-01-20)

$ rustc --version
rustc 1.78.0-nightly (b11fbfbf3 2024-02-03)

But after that, many things are broken, so I had to bump the following packages:

• tokio
• rustls
• wasmtime
• async-compat

I then fixed most of the errors, but I am don't know how to solve the 3 remaining ones about macros:

$ cargo build
   Compiling kannader v0.1.0 (/home/quentin/Documents/dev/kannader/kannader)
warning: the feature `core_intrinsics` is internal to the compiler or standard library
 --> kannader-config/src/lib.rs:1:12
  |
1 | #![feature(core_intrinsics, never_type)]
  |            ^^^^^^^^^^^^^^^
  |
  = note: using it is strongly discouraged
  = note: `#[warn(internal_features)]` on by default

warning: `kannader-config` (lib) generated 1 warning
warning: the feature `core_intrinsics` is internal to the compiler or standard library
 --> kannader/src/lib.rs:1:12
  |
1 | #![feature(core_intrinsics)]
  |            ^^^^^^^^^^^^^^^
  |
  = note: using it is strongly discouraged
  = note: `#[warn(internal_features)]` on by default

warning: unused import: `time::SystemTime`
  --> kannader/src/lib.rs:10:59
   |
10 | use std::{convert::TryFrom, io, path::PathBuf, sync::Arc, time::SystemTime};
   |                                                           ^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(unused_imports)]` on by default

error[E0277]: the trait bound `str: AsContext` is not satisfied
   --> kannader/src/wasm_config.rs:181:1
    |
181 | kannader_config_macros::tracing_implement_host_server!(TracingServer);
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `AsContext` is not implemented for `str`, which is required by `&str: AsContext`
    |
    = help: the following other types implement trait `AsContext`:
              Caller<'_, T>
              StoreContext<'_, T>
              StoreContextMut<'_, T>
              Store<T>
              &wasmtime::store::StoreInner<T>
              &mut wasmtime::store::StoreInner<T>
              &T
              &mut T
    = note: required for `&str` to implement `AsContext`
note: required by a bound in `wasmtime::Linker::<T>::define`
   --> /home/quentin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-17.0.1/src/linker.rs:386:21
    |
384 |     pub fn define(
    |            ------ required by a bound in this associated function
385 |         &mut self,
386 |         store: impl AsContext<Data = T>,
    |                     ^^^^^^^^^^^^^^^^^^^ required by this bound in `Linker::<T>::define`
    = note: this error originates in the macro `kannader_config_macros::tracing_implement_host_server` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0277]: the size for values of type `str` cannot be known at compilation time
   --> kannader/src/wasm_config.rs:181:1
    |
181 | kannader_config_macros::tracing_implement_host_server!(TracingServer);
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ doesn't have a size known at compile-time
    |
    = help: the trait `Sized` is not implemented for `str`, which is required by `&str: AsContext`
    = help: the following other types implement trait `AsContext`:
              Caller<'_, T>
              StoreContext<'_, T>
              StoreContextMut<'_, T>
              Store<T>
              &wasmtime::store::StoreInner<T>
              &mut wasmtime::store::StoreInner<T>
              &T
              &mut T
    = note: required for `&str` to implement `AsContext`
note: required by a bound in `wasmtime::Linker::<T>::define`
   --> /home/quentin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-17.0.1/src/linker.rs:386:21
    |
384 |     pub fn define(
    |            ------ required by a bound in this associated function
385 |         &mut self,
386 |         store: impl AsContext<Data = T>,
    |                     ^^^^^^^^^^^^^^^^^^^ required by this bound in `Linker::<T>::define`
    = note: this error originates in the macro `kannader_config_macros::tracing_implement_host_server` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0061]: this method takes 4 arguments but 3 arguments were supplied
   --> kannader/src/wasm_config.rs:181:1
    |
181 | kannader_config_macros::tracing_implement_host_server!(TracingServer);
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ an argument of type `&str` is missing
    |
note: method defined here
   --> /home/quentin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-17.0.1/src/linker.rs:384:12
    |
384 |     pub fn define(
    |            ^^^^^^
    = note: this error originates in the macro `kannader_config_macros::tracing_implement_host_server` (in Nightly builds, run with -Z macro-backtrace for more info)
help: provide the argument
    |
181 | kannader_config_macros::tracing_implement_host_server!(TracingServer)(kannader_config_macros::tracing_implement_host_server!(TracingServer), kannader_config_macros::tracing_implement_host_server!(TracingServer), /* &str */, kannader_config_macros::tracing_implement_host_server!(TracingServer));
    |                                                                      +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Some errors have detailed explanations: E0061, E0277.
For more information about an error, try `rustc --explain E0061`.
warning: `kannader` (lib) generated 2 warnings
error: could not compile `kannader` (lib) due to 3 previous errors; 2 warnings emitted

[Ekleog]

AFAICT the three issues likely all stem from this line:

kannader/kannader-config-macros/src/lib.rs

Line 463 in 177b7b5

l.define(#link_name, #ffi_name, wasmtime::Func::wrap(&mut *ctx, the_fn))?;

In particular, wasmtime at some point changed its API, so that the Store would be passed as a parameter rather than kept internally within each thing.

Here the add_to_linker method defined by make_host_server seems to already take these arguments:

                ctx: &mut wasmtime::Store<WasmState>,
                l: &mut wasmtime::Linker<WasmState>,

So I'd guess that just replacing this l.define( with l.define(ctx, should hopefully be enough to handle the issue.

Hope that works, and thank you very much for all the updates! :D

[Ekleog]

Oh and as for the macros and why they're here: I would really love to get rid of all these macros altogether, it's one of the most hairy parts of kannader. But unfortunately there's currently no good solution for handling the problem of interfacing between wasm and native.

Unless a solution came in that I didn't hear about, we'll still have to wait for wasm interface types / the wasm component model to come up, at which point we can probably kill all that code with fire :)

[superboum]

So I'd guess that just replacing this l.define( with l.define(ctx, should hopefully be enough to handle the issue.

It appears that the borrow checker does not like that I borrow ctx as mutable at one place, and borrow it somewhere else at the same time:

l.define(ctx, #link_name, #ffi_name, wasmtime::Func::wrap(&mut *ctx, the_fn))?; 

Based on my very limited understanding of both kannader and wasmtime, it seems that Linker::func_wrap is what we want to do here. Especially, based on their example, it appears you can define closures that have as their first parameter a wasmtime::Caller similarly to what Kannader does:

linker.func_wrap("host", "log_str", |caller: Caller<'_, ()>, ptr: i32, len: i32| {
    // ...
})?;

My code is then simply:

kannader/kannader-config-macros/src/lib.rs

Line 463 in e0e9841

l.func_wrap(#link_name, #ffi_name, the_fn)?;

Then it appears the wasmtime::State is bound later. In the example:

let module = Module::new(&engine, wat)?;

// instantiate in multiple different stores
for _ in 0..10 {
    let mut store = Store::new(&engine, ());
    linker.instantiate(&mut store, &module)?;
}

To the best of my understanding, it's the same in your code, for example in kannader/src/wasm_config.rs:

kannader/kannader/src/wasm_config.rs

Lines 94 to 96 in 177b7b5

	linker
	.module(&mut store, "config", module)
	.context("Instantiating the wasm configuration blob")?;

But I am guessing way more things than I am comfortable with here, and I might also have completely broken your WASM implementation without noticing it too 🙈. Let me know!