Switch model system to WASM

[hannobraun]

So far, we've been using dynamically linked libraries to load models into the host application. That works well enough for desktop platforms, but to run Fornjot on the web, with arbitrary models, it is necessary to compile models to WebAssembly and load that into the host application.

If at all possible, WebAssembly should be used for desktop platforms too, so we don't have to maintain two parallel systems. That has the additional advantage of providing sandboxing of models, improving security (the current system is basically unsound by design). Something like Wasmtime or Wasmer could be used, to embed WebAssembly support into the host application for desktop platforms.

[Michael-F-Bryan]

Hi @hannobraun 👋

Something that has gone really well at work is using wit-bindgen for declaring the interface between the host and WebAssembly plugin.

The idea is you write a *.wit file containing all the functions and interface objects your host will provide to the guest (e.g. runtime-v1.wit), plus a *.wit file that defines any functions the plugin will expose to the host (e.g. rune-v1.wit). From there you can use the wit-bindgen CLI to or some procedural macros to generate glue code for both host and guest. If you are familiar with Protocol Buffers, it's a very similar workflow.

The end result is you just need to implement a trait or two and browse your API docs to see what functions are available. We've already got hosts for loading WebAssembly plugins in the browser or using wasmtime from a desktop.

One thing to be aware of is that (by design) WebAssembly can't reference memory from the host, so any data you give plugins access to will need to be copied in. This could get expensive if you are giving plugins direct access to meshes. One solution would be to provide high-level objects that do the expensive manipulation for the plugin.

[hannobraun]

Hey @Michael-F-Bryan, great to see you around here!

Thank you for the comment! This is very useful to me. I had this vague notion that WASM interface types exist, but hadn't looked into it yet. I expect that this information will save me quite a bit of research time, once I'm ready to start working on this.

One thing to be aware of is that (by design) WebAssembly can't reference memory from the host, so any data you give plugins access to will need to be copied in. This could get expensive if you are giving plugins direct access to meshes. One solution would be to provide high-level objects that do the expensive manipulation for the plugin.

That is good to know, thanks. I don't expect that models will have access to meshes, as those are purely for output in Fornjot (modulo #97, but I'm working on that). I do want models to have access to b-rep primitives (#262), but unless the model is emulating some missing CAD kernel feature by creating lots of small edges or faces, I don't expect that to be much data.

We'll see how it shakes out.

[freylint]

Allright. This took me a while, but I've been toying with with trying to port just the fj library to wasm. The problem I'm running into is that the current interface uses non C-Style enums as discussed in the related matrix conversation.

I'll admit I'm no expert on wasm-bindgen or wit, but as far as I can tell using wit-bindgen requires that we have an interface that is callable through the wasm runtime. which leaves the issue of generating that library in the first place.

The current interface doesn't work in WASM as it uses Rust's loaded enums, which don't seem to be trivial to make portable (which is probably why wasm-bindgen doesn't support making bindings to them) without severely losing API ergonomics.

So in order to create a wasm binary for fj we need to be able target a low level representation which doesn't use magic rust features. My experimental implementation which tries to preserve the general structure of the API, but in a polyglot compatible way:

#[derive(Clone, Debug)]
pub enum Shape2d {
    Circle,
    Sketch,
    Difference,
}

#[repr(C)]
#[derive(Debug)]
#[cfg_attr(target_family = "wasm", wasm_bindgen)]
pub struct ShapeHandle2d<'d> {
    /// The primitive shape being referenced
    prim_type: &'static Shape2d,

    /// Reference to the memory of the shape
    /// Gets transmuted to
    data: RawShapeHandle2d<'d>,

    /// Length of memory referenced
    length: usize,
}

/// A raw pointer to data which represents one of the shapes in `Shape2d`
#[derive(Debug)]
struct RawShapeHandle2d<'handle> {
    data: &'handle [u8],
}

impl<'handle> RawShapeHandle2d<'handle> {
    /// Transmute to a a `Circle` without checking validity
    #[inline]
    unsafe fn transmute_circle(self) -> Circle {
        const F64_LEN: usize = std::mem::size_of::<f64>();

        unsafe {
            Circle::new(
                std::mem::transmute(&self.data[..F64_LEN]),
                std::mem::transmute(&self.data[F64_LEN..]),
            )
        }
    }

    /// Transmute to a `Difference2d` without checking validity
    #[inline]
    pub unsafe fn transmute_difference2d(
        self,
    ) -> Difference2d<'static, 'static> {
        const HANDLE_LEN: usize = std::mem::size_of::<ShapeHandle2d>();

        unsafe {
            Difference2d::from_shapes_unchecked((
                std::mem::transmute(&self.data[..HANDLE_LEN]),
                std::mem::transmute(&self.data[HANDLE_LEN + 1..HANDLE_LEN * 2]),
            ))
        }
    }

    /// Transmute to a `Sketch` without checking valifity
    #[inline]
    pub unsafe fn transmute_sketch(self) -> Sketch {
        const PTR_LEN: usize = std::mem::size_of::<*mut [f64; 2]>();
        const USIZE_LEN: usize = std::mem::size_of::<usize>();
        const ATOMIC_USIZE_LEN: usize =
            std::mem::size_of::<atomic::AtomicUsize>();
        const COLOR_LEN: usize = std::mem::size_of::<[u8; 4]>();

        const LENGTH_OFF: usize = PTR_LEN + 1;
        const CAPACITY_OFF: usize = LENGTH_OFF + USIZE_LEN + 1;
        const RC_OFF: usize = CAPACITY_OFF + USIZE_LEN + 1;
        const COLOR_OFF: usize = RC_OFF + ATOMIC_USIZE_LEN + 1;

        unsafe {
            Sketch::new(
                std::mem::transmute(&self.data[..PTR_LEN]),
                std::mem::transmute(
                    &self.data[LENGTH_OFF..LENGTH_OFF + USIZE_LEN],
                ),
                std::mem::transmute(
                    &self.data[CAPACITY_OFF..CAPACITY_OFF + USIZE_LEN],
                ),
                std::mem::transmute(
                    &self.data[RC_OFF..RC_OFF + ATOMIC_USIZE_LEN],
                ),
                std::mem::transmute(&self.data[COLOR_OFF..]),
            )
        }
    }
}

(I'm aware of the MANY bugs with this implementation, I just wanted something that would compile and demonstrate my approach. I have never written code this low level before 😧)

This "creates" a very low level private implementation that deals with pointers and memory offsets and a higher level implementation that deals in shapes, but still somewhat resembles the current API. I think I could then create the existing API on top of this for people building as an rlib.

** It seems that we're either going to have to maintain .wit assemblies or change the interface so much that wasm-bindgen would support it anyway. **

This whole experiments makes me think a hypothetical distribution platform for end users would be better off distributing the models as source instead and embedding a compiler in end user applications if API ergonomics are a higher priority than portability.

This is a tough nut to crack for me and I'm struggling to do it, but it's improving my knowledge is expanding quite satisfyingly while doing so, so I'd be happy to keep going with this approach, but it seems like a nightmare.

[hannobraun]

Thank you looking into this, @freylint! I agree that this is a nightmare. Seems unmaintainable 😄

I don't have time right now to look into this myself, unfortunately, so all I can do is pose a few questions:

• What about fp-bindgen? It looks like it might support enums (example).
• Is there another way to express what the fj crate needs to express, without using enums?
• Maybe wanting to use WASM is misguided in the first place?

I don't expect you to answer these, just trying to contribute to the discussion. Looks like fp-bindgen might be worth a look though.

[Protryon]

I've worked with some complex WASM integrations professionally. I don't know if I'll have the time to integrate with stuff here, but the approach I'd roll with, especially to support arbitrary languages:

• Define a stable C API for fornjot -- C isn't a language, it's a standard now :P
• Define a Rust API on the C api that can be compiled from WASM land
  I would just use wasm32-unknown-unknown target, avoiding WASI since we want a sandboxed environment and emscripten since we don't need JS integration.

What this might look like, project wise:

• Define a C header(s) for the common API
• I've not worked with an engine other than V8 (and I never directly targeted WASM there, mostly working on WASM-impl side), but presumably you can hook up those native methods to Rust directory, or if you want to be a bit more robust, to native C calls that call Rust. This allows API parity in native space if needed.
• For any language support is wanted in, write bindings to C apis. Python, C++, Rust, etc. Rust being the most obvious, would just use unsafe code in WASM land to call the C API functions.

[hannobraun]

Thank you for this information, @Protryon! Do you have any experience with or opinion on fp-bindgen or wit-bindgen (or WebAssembly Interface Types in general)?

It sounds to me like WIT might be a good solution for this kind of thing in the future. fp-bindgen presents itself as a short-term alternative to WIT, while that isn't quite ready yet. It has the advantage of supporting arbitrary Rust types, which the other options don't do.

I do think that the approach you present here sounds perfectly reasonable, but also somewhat labor-intensive. Both in terms of implementation and ongoing maintenance. I'm wondering how the C-based approach compares to WIT, and how both fit into the larger context of WebAssembly bindings.

fp-bindgen sounds very attractive to me as a "good enough for now" solution, as its full support for Rust types could make integration into Fornjot relatively straight-forward, at the cost of making making support for other languages impractical. That could be the right trade-off right now.

[Protryon]

fp-bindgen looks like it's still in its infancy, so you might get a hiccup and fail early (and hopefully fast if at all). wit_bindgen seems similar.

The C-based approach is indeed more labor intensive than a drag-and-drop solution at first glance. With the lack of maturity and broader language support, I would think they are about the same with some trade-offs in outcome.

[hannobraun]

fp-bindgen looks like it's still in its infancy, so you might get a hiccup and fail early (and hopefully fast if at all). wit_bindgen seems similar.

I've done some limited experimentation with fp-bindgen, so far without getting it fully working (I haven't spent a lot of time on this). It certainly seems like it needs to be held in just the right way to get it to work, which makes sense, given that it's new.

The C-based approach is indeed more labor intensive than a drag-and-drop solution at first glance. With the lack of maturity and broader language support, I would think they are about the same with some trade-offs in outcome.

Yeah, that sounds about right. For now, I'm open to all the approaches mentioned so far.

Maybe the C-based approach warrants some more exploration. I'm not up-to-date on Rust/C FFI and what tools are available these days. Could be, that my mental model is outdated, and it's not as much work as I fear.

[hannobraun]

We've been having some discussions in the Matrix channel lately, and reflecting on those, I've changed my opinion about this issue. I previously saw a WASM-based model system as a mid-term priority, and something I might have wanted to work on myself in the not too distant future. While I still think that this is likely the right direction long-term, I no longer see this issue as a priority.

Before I explain my reasoning, let me recap the benefits that I see in a WASM-based model system:

• Security: The current approach based on dynamic libraries is insecure by design. It uses unsafe code that can't be sound, unless we assume that models can be trusted fully. And we're talking about memory safety here. In addition, models are written in a general-purpose language, and can do whatever they want to your filesystem, for example. Running models in a sandbox would solve those problems.
• Multi-language support: I would like to see a future where Fornjot models can be written in multiple languages, with the ability to mix and match between them. For example, it shouldn't be a problem to use a component written in Rust, in a model written in some scripting language. WASM would be a step towards that, and there's even existing infrastructure (like WAPM) that could be useful for us.
• Browser support: I'd like to be able to embed Fornjot models into websites, without the need for hosting a special backend. If Fornjot models were just WASM modules, a Fornjot app that itself is compiled to WASM, could load them from within the browser.

I think we can ignore security for now. Fornjot doesn't even have the most basic modeling features implemented. As long as it isn't really useful to anybody, I don't think we have to worry about people publishing malicious Fornjot models. Likewise, as long as we don't have a foundation of useful modeling features, it makes no sense to expose what little we have to multiple languages. Multi-language support is a nice-to-have for (potentially much) later.

That leaves browser support, which is still something that I'd personally like to see. However, I think I had developed a bit of tunnel vision in that regard. WASM-based modules aren't necessary for browser support. For that, we can also use an approach where models are regular Rust applications that use Fornjot as a library, and compile that whole thing to WASM.

This wouldn't be a practical model for the general case, due to long compile times (I've tried). But it would work for deploying your finished model to a website. I'm going to write about that in other issues, as it would be beside the point here.

So where does that leave us? Here's the current situation, from my perspective:

• I myself won't work on this any time soon, focusing instead on basic CAD features and other work that will have a more direct impact on making Fornjot useful to people.
• I still welcome experimentation by contributors. As I said above, I still see WASM-based modules as the likely long-term future for Fornjot.
• However, based on the previous discussion here, I might be wary of merging anything right now that introduces too much complexity, and too much of a maintenance burden. Especially as long as there's some vague promise that WIT or whatever else could make things easier down the line.

That last item doesn't mean I won't merge anything that takes Fornjot into this direction, but I don't want anyone to feel like they're wasting their time either. If in doubt, feel free to ask here, or the Matrix channel, or Discussions.

As always, feedback is appreciated!

[Michael-F-Bryan]

As mentioned on Matrix, I think focusing on the CAD features and general workflow is the right decision here.

The reality is that we've already got a model system. Sure, it may not be how things are done later on, but for now it works well enough that other parts of the project can make process. Changing how models and the host interact is a fundamental part of the project's architecture (#804), whereas switching from extern "C" to WebAssembly is "just" an implementation detail. Furthermore, we've already got nice abstractions in place that will help with the transition process.

---

Regarding the WebAssembly model implementation I'd like to share my experiences doing a very similar thing for work.

For context, we make an application that lets you define and execute data processing pipelines, possibly including ML models, where each operation ("processing block") is some sort of plugin (in this case, WebAssembly modules). We've gone through roughly three different implementations for this plugin system and I learned loads in the process.

Initially, each pipeline was a Rust crate that we generated from some intermediate representation, where each processing block was a Rust crate added to Cargo.toml as a dependency. The final pipeline was compiled to WebAssembly and would communicate with the host via extern "C" functions we'd written manually (essentially, @freylint's approach). This kinda worked, but we ended up having 2kloc of unsafe Rust that was a pain to write/maintain, and like a lot of extern "C" code it was very easy to mess up things things like struct layouts and argument passing.

For the second and third implementations, each processing block is now its own WebAssembly module and it communicated with the host using wit-bindgen. It was unbelievable how much of an improvement this made to developer experience and productivity. By just needing to define the interface and having a tool generate the glue code for you, it makes it super easy to iterate, and writing a host is just a case of implementing a trait. 10/10 would recommend in the long term.

I like the init() approach. It gives each model an opportunity to ask the host about things and register interest in certain events while also leaving room for evolution. Just be aware that the "chattier" your guest-host interface, the more painful it'll be to maintain a hand-written extern "C" API.

This is probably more relevant to #804, but one suggestion is to give model developers a "seam" they can use during testing instead of APIs that talk to the host directly. It's hard to explain in words, so here's some code. In our second iteration, we made the mistake of creating free functions for talking with the host (getting arguments, etc.) and as a result it became hard to write unit tests for those interactions. Writing tests that transitively call these extern "C" functions will lead to linker errors, so we ended up only testing the core logic and the code at the periphery like reading arguments and validating inputs - often where the majority of your bugs are - was left untested.

If you want to see an example of what this looks like, we define the host-guest interface using WIT files and have a repo containing all the "processing block" implementations we provide.

The hotg-ai/proc-blocks repo also contains a host implementation (2nd iteration, 3rd iteration) which we use for getting a proc-block's metadata and running its graph() function (i.e. given these arguments, what do your input tensors and output tensors look like?).

For future reference, Rust-style enums are a first-class citizen in WIT. Just keep in mind that C-style enums are declared with the enum keyword while Rust-style enums use the variant keyword (example).

Some things to keep in mind while iterating on the model API:

• Translating from extern "C" to something based on WIT files will have an impact on the way your API is structured because WebAssembly can't reference host memory by design. It's not as big a deal as you would think because of things like the reference types proposal (which WIT currently polyfills using IDs) and it'll only become a performance issue when you are passing meshes around instead of high-level shapes - just something to keep in mind
• WIT and wit-bindgen are still under active development, so you might want to pin everyone to a specific version of the wit-bindgen crate because things like the glue code generated for calling and passing arguments (I guess the best name for it is "ABI") might change over time. That's only a concern for when you have external users though because developers will already be compiling from source and all the model implementations are part of the Fornjot repo
• It's up to you, but we decided that proc-blocks should be compiled as wasm32-unknown-unknown instead of wasm32-wasi. This means the only way the guest can interact with its environment is via the API you provide, which is great for security and control, but also adds limitations on the model (can't interact with the file system, need to use a logger instead of println!(), etc.).

[hannobraun]

Thank you for posting this, @Michael-F-Bryan! Very valuable information.

I'd like to say that I'm fully on board with what you wrote here, and want to mirror some of the comments I made in the Matrix discussion:

• Glad to hear that wit-bindgen does indeed support enums! I think I figured out where my own confusion on the subject came from: wasm-bindgen doesn't support them, I got confused by WIT's enum/variant distinction, and that probably combined in my mind into the false impression that WIT doesn't support them either. Glad to hear I'm wrong!
• While it sounds like WIT is the way to go long-term, I think our API surface is small enough right now, that the extern "C" approach is fine. I would welcome it, if someone wants to look into WIT integration, but it doesn't need to be a part of the initial implementation.

It's up to you, but we decided that proc-blocks should be compiled as wasm32-unknown-unknown instead of wasm32-wasi. This means the only way the guest can interact with its environment is via the API you provide, which is great for security and control, but also adds limitations on the model (can't interact with the file system, need to use a logger instead of println!(), etc.).

Thanks for the note. I think WASI could be very beneficial for some Fornjot models, as it would allow model authors a lot of flexibility doing things that Fornjot doesn't support yet. Based on some uses of OpenSCAD I've seen in the wild, I assume this will become relevant sooner or later.

For an initial implementation, I think wasm32-unknown-unknown makes most sense. We could explore how to support wasm32-wasi later. Long-term, giving the user control over the permissions that a model gets (maybe taking some inspiration from Android/iOS for the UI) might be best, but I think it'll be a while before it makes sense to worry about that.

[MartinKavik]

I've done some research & experimentation with with different Wasm runtimes:

1. Wasmtime
   • It doesn't support browser environment at all. (Please correct me if I'm wrong.)
2. Extism
   • It's a plugin system based on Wasm.
   • It has only a JS/TS host in a browser. (Rust host is based on Wasmtime if I remember correctly.)
3. Wasmi
   • Wasm interpreter.
   • Can be run inside Wasm (i.e. it supports Rust host in the browser).
   • Successfully tested inside a MoonZoon app (= Wasm module) with a Wasm plugin compiled for the wasm32-unknown-unknown target.
   • 2.3M downloads on crates.io, seems to be actively maintained by Parity.
   • It doesn't support re-entrancy (yet), i.e. you can't call a plugin function from a host function.
   • Some Wasm proposals/features aren't planned or implemented.
   • No bindgen supports Wasmi (please correct me if I'm wrong).
4. Wasm3
   • Wasm interpteter.
   • Written in C, Rust bindings are available.
   • Should support Rust/Wasm host, I haven't tested it.
5. Wasmer
   • It seems to be the main "competitor" to Wasmtime.
   • Supports Rust host in Wasm thanks to wasm-bindgen (a Rust<->JS bridge). But the support seems to be a better PoC rather than a production ready code.
   • I've managed to make it work with my test MoonZooon app.

Protocols / binding generators:

1. waPC
   • It looks like a simpler protocol for sending bytes between the host and plugins through functions identified by strings.
   • Supports Wasm3 and Wasmtime according to the Rust docs.
2. wasm-bindgen
   • Rust <-> Javascript/Typescript bridge.
   • I think it's currently the only Rust/JS bindgen used by Rust frontend frameworks.
   • Rust-typed JS and Web API are defined in wasm-bindgen-related crates like js-sys, web-sys, wasm-bindgen-futures, etc.
3. fp-bindgen
   • Generates Rust/Rust or Rust/Typescript bridge.
   • Uses Wasmer and MessagePack.
4. wit-bindgen
   • It started like a bindgen generating bridges from WIT definitions/model but it seems to be migrating to a Component Model that is inspired/based on WIT model.
   • It's developed by Bytecode Alliance so it looks like the development and design focus on Rust and Wasmtime.
   • It can generate only JS host for the browser environment.
   • Bonus: see cargo component
5. wai-bindgen
   • Forked wit-bindgen before their original authors started rewriting it to Component Model.
   • It was forked to add Wasmer support.
   • It looks like @Michael-F-Bryan is maintaining the bindgen and related Rust crates.

---

So... I think the best combination for Fornjot (and maybe also MoonZoon) is currently Wasmer + wai-bindgen. Both Wasmer browser support and wai-bindgen + WIT definitions are a bit unstable but they make the plugin system development much easier and we'll be able to support other languages. I saw at least Python and Javascript support in the Wai repo - both could be useful for "real-time" Fornjot demos and modelling.

---

I've tried to use Wasmer inside a MoonZoon app with both hand-written code and a bridge generated by wai-bindgen and related macros from a *.wit file. I was successful, but I had to fork some Wasmer-related repos to make it work.
Problems I encountered while integrating Wasmer:

1. Submodules in the Wasmer (?) repo are broken - it's not possible to link the library in Cargo.toml as a dependency with git = .
2. wasmer cargo feature wasm-types-polyfill has to be enabled on the host to prevent constant failing because of incompatible exported function types.
3. Type checker doesn't seem to work correctly or it reads all function parameters and return types as i32 by default.
4. Some host functions in Wasmer/Wai are not implemented because they don't make sense in the browser context - e.g. functions data_unchecked, data_unchecked_mut or Module::from_file. But bindgen uses some of them.
5. Rust guests can't be compiled and instantiated on browsers like Chrome when they are larger than 4 KB. Synchronous comp./instan. Wasm modules larger than 4 KB is possible only on Web Workers. It means we have to use async WebAPI methods to start Wasm guests. And because the main browser thread can't be blocked we have to modify the Wasmer API to have async functions as well. Also a function Module::from_reponse(web_sys::Reponse) would be very nice to have. These changes would introduce API differences between the Wasmer native and browser versions - it would have to be resolved somehow. And then all generators would have to be updated.
6. There are some hidden errors, unnecessary unsafe blocks or unused variables in the Wasmer browser version but I think it's just WIP code.
7. The wai crate uses wasmer as a dependency with default cargo features. It fails to compile in Wasm because of incompatibilities introduced by those enabled features.

@Michael-F-Bryan I would be glad for your opinions or let me know where/if I should create a PR or something like that.

---

Once Wasmer fully supports Rust-in-browser then I would like to try compile Fornjot to Wasm module to find out which dependencies aren't Wasm-compatible or if there are too large dependencies.

Another potential problem is parallelization. The only way to support multithreading in a browser is to leverage Web Workers and don't use blocking calls at all (only async functions or callbacks can be used). There are some Rust multithreading libraries trying to abstract out Web Workers and SharedArrayBuffer and other things (that are finally supported by browsers again and some of them usable at least on a nightly Rust). However, when I was testing some of them months/years ago I wasn't very successful with integrating them. I hope it's better situation now but I would recommend to implement some async-related stuff like #1327 so we have something to test while we'll be testing Fornjot in the browser environment.

[fschutt]

@MartinKavik If you have issues with Wasmer, it would be great to file them on the wasmer repo, so we can track them.

However, I've tested (1) with

[dependencies]
wasmer = { git = "https://github.com/wasmerio/wasmer" }
wasmer-wasi = { git = "https://github.com/wasmerio/wasmer" }
wasmer-vfs = { git = "https://github.com/wasmerio/wasmer" }

and

[dependencies]
wasmer = "3"
wasmer-wasi = "3"
wasmer-vfs = "3"

... and it does work? I'm not sure what the submodule errors you're getting look like. We did have submodule errors about a month ago, but they should have been removed on the latest master.

[hannobraun]

Thank you for that thorough overview, @MartinKavik! And thanks for pitching in, @fschutt. Nice to see you here!

1. Wasmtime

   * It doesn't support browser environment at all. (Please correct me if I'm wrong.)
   

My assumption so far was that we'll have different code paths for running inside or outside of a browser. Basically, I assumed we'd have Wasmtime or Wasmer as a replacement for the current libloading-based approach, but use different code for web support¹.

But if Wasmer can handle both scenarios, all the better!

So... I think the best combination for Fornjot (and maybe also MoonZoon) is currently Wasmer + wai-bindgen. Both Wasmer browser support and wai-bindgen + WIT definitions are a bit unstable but they make the plugin system development much easier and we'll be able to support other languages.

Sounds reasonable!

Please note, this issue is about migrating the model system to WASM. That's a part of browser support, but it doesn't need to include browser support in the initial version. Of course, we wouldn't want to take an approach that would make adding browser support later any harder than necessary. Just saying, we don't need to solve all the problems at once. (Also see #816.)

Once Wasmer fully supports Rust-in-browser then I would like to try compile Fornjot to Wasm module to find out which dependencies aren't Wasm-compatible or if there are too large dependencies.

Most of Fornjot already compiles to WebAssembly, and this is part of the CI build. Notable exceptions include fj-app (which deals with configuration files and command-line arguments, so inherently it's not WASM-friendly) and fj-host (which currently loads dynamic libraries, but that should become more tractable once it loads WASM modules instead).

Footnotes

1. I assumed that eventually, we'd be able to dynamically load WASM-based models based on the standard web APIs, but initially it would be perfectly fine by me if we couldn't do that at all, and web support meant just compiling a model + Fornjot as a single WASM module. ↩

[MartinKavik]

@fschutt :

If you have issues with Wasmer, it would be great to file them on the wasmer repo, so we can track them.

I'd love to do it now but the end of the month is coming up so I have to do some paid work as well=> I'll try to look at it during the following weeks.

... and it does work? I'm not sure what the submodule errors you're getting look like. We did have submodule errors about a month ago, but they should have been removed on the latest master.

I've written "Submodules in the Wasmer (?) repo are broken" with a question mark because I've forked ~4 Wasmer-related repos and don't remember where the problem appeared.

---

@hannobraun:

I think Wasmer + WIT is the only reasonable choice on the browser now and the testing is quite easy for me because I can just add Fornjot into a basic MoonZoon app as a dependency and run the command mzoon start from my terminal because all tooling including auto-reload, a dev server and Wasm optimizations are already present in MoonZoon CLI.
And I assume once Fornjot works in Wasm with Wasm models then it should be pretty easy to make it work on a native machine, too.

So as the next step I'll try to push the Wasmer fixes a bit once I find some time and then experiment with Fornjot compilation in the browser Wasm to have an idea how much work would be needed to make it compatible.

[hannobraun]

Sounds great, @MartinKavik. Thank you!

[freylint]

I'd be happy to help contribute to this feature. I'm only a hobbyist programmer, but I've been around for a couple years. I'd need mentorship / direction to really be able to tackle this problem.

[hannobraun]

Thank you, @freylint! I'm happy to help, but my own knowledge is mostly limited to what others have posted here. If I were to tackle this myself, I'd just try out the approach that @MartinKavik just suggested.

Maybe @MartinKavik has some input for you?

[MartinKavik]

I've just merged an example with Wasmer to the MoonZoon repo so we have a testable example.

You can run & test the example this way:

1. Clone/Fork the repo https://github.com/MoonZoon/MoonZoon
2. rustup update stable
3. rustup target add wasm32-unknown-unknown
4. cargo install cargo-make
5. Go to examples/wasm_components
6. makers mzoon start -o
7. Wait until you see the app running in your new browser tab and then open web dev console.
8. Go to examples/wasm_components/frontend/components/calculator
9. cargo build -r
10. Drag&Drop calculator.wasm from examples\wasm_components\frontend\components\calculator\target\wasm32-unknown-unknown\release onto the dropzone in the running app in the browser.

wasm_component_moonzoon_demo.mp4

Once we resolve all todos marked @TODO-WASMER in the code in the example and in 2 related forked Wasmer repos (= example's deps) then Wasmer should be ready for Fornjot with a bit of luck :)

If you encounter some obstacles related to the MoonZoon, just join MoonZoon Discord and write me.

[hannobraun]

Extism, a WASM-based universal plugin system, was announced today: https://extism.org/blog/announcing-extism/

It seems to be based on Wasmtime and has a browser runtime. I haven't looked into it more deeply than that, but it does sound interesting! Being a plugin system and not just a WASM runtime, it could probide useful stuff that Fornjot needs. On the other hand, it might force us to do things in a way we don't like. Hard to say without taking a closer look, but it might be worth doing so.

[MartinKavik]

Extism, a WASM-based universal plugin system, was announced today

https://discord.com/channels/1011124058408112148/1011124061100843020/1043580405514784870

[hannobraun]

Oh, so the browser runtime is not the same as the Rust runtime? Doesn't seem suitable to our purposes then.

[MartinKavik]

A Wasm host (e.g. Rust in a browser) is basically an edge-case from the point of view of these plugin systems, unfortunately. Javascript is meant by browser runtime in almost all cases. I was able to run only Wasmi and modified Wasmer in a Wasm host.

[silesmo]

Hello!
Just wanted to mention that I have been working on this throughout the week and it's mostly working now.
I went ahead and implemented it with Wasmer and WAI. There are some minor kinks that I'm still working on resolving.
The primary one being the fact that WAI doesn't support recursively referring to it self and as there is no way to provide a reference to it as far as I know (with WAI). Thus we will have to come up with something else.
What I'm currently thinking and have mostly implemented is passing shapes and shapes2d in a struct and then have the structs inside the shape enum (or variant in WAI) store an index for the shape/shape2d in the list.

It would then look something like this, where ShapeData is what would be returned instead of Shape from the shape function:

pub type ShapeHandle = u32;

pub struct ShapeData {
  pub shapes2d: Vec<Shape2d>,
  pub shapes: Vec<Shape>,
}

pub struct Group {
  pub a: ShapeHandle,
  pub b: ShapeHandle,
}

Which would be this in WAI:

type shape-handle = u32

record shape-data {
    shapes2d: list<shape2d>,
    shapes: list<shape>
}

record group {
    a: shape-handle,
    b: shape-handle
}

What do you think about that @hannobraun? Or do you have another approach?

[hannobraun]

Thank you for your work, @silesmo!

I haven't worked with WAI or WIT before, so I can't provide much guidance on that side. But in general, I can say that your proposal looks reasonable. One thing that comes to mind, is whether it's a good idea to have a single ShapeHandle type that can refer to both Shape2d and Shape. But that's the kind of thing we can think about later.

As a general approach, I suggest to just make it work and worry about the details later. Having WASM support will be a big step in the right direction, and once all of that exists and is merged, we will be in a better position to figure out better usage patterns.

[voxpelli]

Random note: Could perhaps be interesting to look at how Envoy added extensibility through WASM: https://tetrate.io/blog/wasm-modules-and-envoy-extensibility-explained-part-1/

[silesmo]

Okay, I will go ahead with this then! I have one handle for shape and one for shape2d I just included one as part of the example.
Will be posting a PR this week.

[hannobraun]

Sounds great, @silesmo. Looking forward to the PR!

[silesmo]

Hello again @hannobraun. Life got a bit in the way last week so didn't get the time to finish up the PR. By the looks of it I won't have the time until Tuesday next week. Just wanted to keep you in the loop. Sorry for the delay!

[hannobraun]

Thanks for letting me know, @silesmo. Take your time!

[IamTheCarl]

This subject came up over in #1569.
Capturing panics from a dynamic libraries brings in some spookiness, so I suggested an alternative way to get a model's data from model code into Fornjot.

The benefits and reasoning behind this design are explained here.

I'm going to start on that experimental prototype. I won't be implementing it for WASM at this time (you all seem to know more about that than I do anyway) but I do think I'll rework the current dynamic library system to pass a pointer to a block of memory with the serialized format contained within it.

[hannobraun]

This issue is no longer applicable. See A New Direction for context.

Thanks everyone, for all the input here! All remaining Fornjot components already compile to WebAssembly and it should be possible to use them as libraries in WASM. The remaining parts of this issue are no longer in scope for the project.