working with strings

[proohit]

Summary

I've been wondering how to handle strings in python + wasm (with wasmer). There are proposals like reference types and interface types and low-level examples manipulating memory to handle strings. As wasmer implements at least reference types, is there an easy way to e.g. pass strings to exported wasm functions?

Additional details

I was trying this out with wasm-bindgen and implemented the wasi example from wasmtime, but transformed the wasm module into a reactor module. This very easy function copies one given file (param1) to another location (param2).

use std::fs;
use std::io::{Read, Write};

fn process(input_fname: &str, output_fname: &str) -> Result<(), String> {
    let mut input_file = fs::File::open(input_fname)
        .map_err(|err| format!("error opening input {}: {}", input_fname, err))?;
    let mut contents = Vec::new();
    input_file
        .read_to_end(&mut contents)
        .map_err(|err| format!("read error: {}", err))?;

    let mut output_file = fs::File::create(output_fname)
        .map_err(|err| format!("error opening output {}: {}", output_fname, err))?;
    output_file
        .write_all(&contents)
        .map_err(|err| format!("write error: {}", err))
}

use wasm_bindgen::prelude::*;

#[wasm_bindgen]
pub fn load_model(input_fname: &str, output_fname: &str) -> () {
    if let Err(err) = process(&input_fname, &output_fname) {
        eprintln!("{}", err)
    }
}

Compiling with cargo rustc --release --target wasm32-wasi -- -Z wasi-exec-model=reactor results in a wasm module which includes wasm-bindgen imports and exports, including memory manipulation functions:

  (import "__wbindgen_placeholder__" "__wbindgen_describe" (func $_ZN12wasm_bindgen19__wbindgen_describe17h1713c9eafed8f0bcE (type $t0)))
  (import "__wbindgen_externref_xform__" "__wbindgen_externref_table_grow" (func $_ZN12wasm_bindgen9externref31__wbindgen_externref_table_grow17ha6f4f5bae0e7b807E (type $t6)))
  (import "__wbindgen_externref_xform__" "__wbindgen_externref_table_set_null" (func $_ZN12wasm_bindgen9externref35__wbindgen_externref_table_set_null17h2ad33b4d577a8fa6E (type $t0)))
  (import "wasi_snapshot_preview1" "fd_close" (func $_ZN4wasi13lib_generated22wasi_snapshot_preview18fd_close17h637d5166044e9087E (type $t6)))
  (import "wasi_snapshot_preview1" "fd_read" (func $_ZN4wasi13lib_generated22wasi_snapshot_preview17fd_read17h4151fc4f09f71f43E (type $t7)))
  (import "wasi_snapshot_preview1" "fd_write" (func $_ZN4wasi13lib_generated22wasi_snapshot_preview18fd_write17h7c5a0bb2c5af3798E (type $t7)))
  (import "wasi_snapshot_preview1" "path_open" (func $_ZN4wasi13lib_generated22wasi_snapshot_preview19path_open17hc57443ec1b90b55dE (type $t8)))
  (import "wasi_snapshot_preview1" "fd_prestat_get" (func $__wasi_fd_prestat_get (type $t3)))
  (import "wasi_snapshot_preview1" "fd_prestat_dir_name" (func $__wasi_fd_prestat_dir_name (type $t5)))
  (import "wasi_snapshot_preview1" "proc_exit" (func $__wasi_proc_exit (type $t0)))
  (import "wasi_snapshot_preview1" "environ_sizes_get" (func $__wasi_environ_sizes_get (type $t3)))
  (import "wasi_snapshot_preview1" "environ_get" (func $__wasi_environ_get (type $t3)))
...
  (export "load_model" (func $load_model.command_export))
  (export "__wbindgen_describe_load_model" (func $__wbindgen_describe_load_model.command_export))
  (export "__externref_table_alloc" (func $__externref_table_alloc.command_export))
  (export "__externref_table_dealloc" (func $__externref_table_dealloc.command_export))
  (export "__externref_drop_slice" (func $__externref_drop_slice.command_export))
  (export "__externref_heap_live_count" (func $__externref_heap_live_count.command_export))
  (export "__wbindgen_exn_store" (func $__wbindgen_exn_store.command_export))
  (export "__wbindgen_malloc" (func $__wbindgen_malloc.command_export))
  (export "__wbindgen_realloc" (func $__wbindgen_realloc.command_export))
  (export "__wbindgen_free" (func $__wbindgen_free.command_export))

So given this simple python program to call this function, how can I interact with it so that it passes parameters correctly? Note that this program works, when the exported functions has hard coded paths instead.

from wasmer import engine, wasi, Store, Module, ImportObject, Instance
from wasmer_compiler_cranelift import Compiler
import os

__dir__ = os.path.dirname(os.path.realpath(__file__))
wasm_bytes = open(__dir__ + '/smartcore_wasi_lib.wasm', 'rb').read()

store = Store(engine.JIT(Compiler))

module = Module(store, wasm_bytes)

wasi_version = wasi.get_version(module, strict=True)

wasi_env = \
    wasi.StateBuilder('smartcore-wasi'). \
        preopen_directory("."). \
        finalize()

import_object = wasi_env.generate_import_object(store, wasi_version)

instance = Instance(module, import_object)
# works with no parameters, if paths are hard coded
# instance.exports.load_model()
instance.exports.load_model('test1.txt', 'test2.txt')

[Hywan]

Hello,

I've never tried to use wasm-bindgen outside JS, and I'm sure it's not intented to work outside the JS world.
But if you would like to try, then you need to implement host functions for __wbindgen_*. See this example to learn how, https://github.com/wasmerio/wasmer-python/blob/ca45878c545b2280aba0000515f43c814916baca/examples/imports_function.py.
Or check the documentation, https://wasmerio.github.io/wasmer-python/api/wasmer/#wasmer.ImportObject.

Does it help?

[proohit]

Thanks for your reply!

I think implementing them with ’wasm-bindgen’ is one attempt. Is there a suggested way to work with strings in python + wasmer? Or are we waiting for interface-types to be more mature?

[Hywan]

Ideally, you should wait on WIT to be more mature, but it won't happen before at best one year.

Personally, I would just write a allocate and deallocate functions, and then using them and passing pointers manually. It's solid and easy to do.

[Hywan]

Check this example, https://github.com/wasmerio/wasmer-python/blob/ca45878c545b2280aba0000515f43c814916baca/examples/appendices/greet.py (with the Rust implementation, https://github.com/wasmerio/wasmer-python/blob/ca45878c545b2280aba0000515f43c814916baca/examples/appendices/greet.rs).

[proohit]

That's exactly what I was looking for, thanks!!