Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Land WASI Preview 2 support into Wasmtime #6370

Closed
7 of 11 tasks
pchickey opened this issue May 11, 2023 · 1 comment
Closed
7 of 11 tasks

Land WASI Preview 2 support into Wasmtime #6370

pchickey opened this issue May 11, 2023 · 1 comment

Comments

@pchickey
Copy link
Contributor

pchickey commented May 11, 2023

We have been working on a prototype of what WASI Preview 2 support will look like in Wasmtime for 7 months now! https://github.com/bytecodealliance/preview2-prototyping/

The work is not yet totally complete, but I believe that it is mature enough that we should start landing it into this repository, and keep iterating on it here. It was included in the March 28 tooling compatibility matrix, but it turns out we skipped compatibility with the wasmtime 8.0 release due to some bindgen changes we needed. I'd like to get everything landed and part of the regular Wasmtime release cycle for 10.0, which will cut at the beginning of June.

Here is my rough plan for how we will land changes. These plans are open to changes so please leave your feedback!

  1. Move the adapter (found at the root of the p2p repo) to crates/wasi-preview1-component-adapter. We will bring its child proc-macro crate byte-array, and the test harness validate, in as well underneath that directory. Add building, validating, and publishing the adapter binary (for both commands and reactors) to the wasmtime repository's CI. Runtime (integration) tests will be added in the subsequent steps.
  2. Restructure the test-programs crate to support building both modules and components (using the adapter from step 1). Keep all the existing tests of wasi-common (which covers both the cap-std-sync and tokio backends) intact, but refactor like we did in p2p to make the structure of how tests are run driven by hand-written code, rather than packing a ton of magic into the macros. The components will be built, but not actually executed yet.
  3. Land the contents of preview2-prototyping's wasi-common crate under wasmtime-wasi in the pub mod preview2 space, and behind the (enabled by default) preview2 cargo feature. Leave this repository's wasi-common, wasi-cap-std-sync, and wasi-tokio crates untouched. They will continue to provide all preview 1 functionality for the moment. The preview 2 functionality will be passing all tests that it does in p2p through the test-programs (at the moment this means poll-oneoff and some trailing slashes behavior is broken), and this PR will port in the other tests from the p2p repo as well.

At this point we can say the wasmtime project has preview 2 support! It is, however, behind a different WasiCtx api (with some changes to the builder, and the way to add it to a linker), and is totally separate from the preview 1 support.

The next steps are:

  1. Finish the development work I have planned to straighten out the poll-oneoff / streams / async vs sync embedding design in preview 2, which is currently not passing all its tests. At this point, we can just ignore some trailing slash behavior in the test suite, but the stuff everyone really cares about ought to be done.
  2. Land @rvolosatovs 's work on a host-side preview 2 wit-bindgen to preview 1 wiggle adapter. Hook this up to the test suite, passing all the same tests as the legacy implementation, without using the wasi-preview1-component-adapter or any component-model features in wasmtime.
  3. Land preview2-prototyping's wasmtime-wasi-sockets crate into this repository. This crate is separate from, but depends upon, what will land in wasmtime-wasi::preview2. It need to be integrated with the poll oneoff design fixes, and it needs a test suite, in order to land.
  4. Port the wasmtime-wasi-http crate to use the poll-oneoff fixes above, making it compatible with components instead of just modules]
  5. Bring wasmtime-wasi::preview2's docs up to Wasmtime project standards.

Once the poll design changes and host-side adapter are landed, we can start working on transitioning the legacy implementation out of wasmtime:

  1. Move all of wasmtime-wasi's existing preview 1 support behind pub mod preview1_legacy and the off-by-default preview1_legacy cargo feature. Move the preview2 module out to the root. The new host-side adapter interface to provides preview 1 support to all users, except those who have regressions and need to use the legacy support for a limited time.
  2. The known regressions for preview 2 support include some trailing slash filesystem behaviors, and the support for preview 1's late-added sock_* functions. Presently, the sock functions have no tests in the tree, or anywhere else I can find, so I don't have a way to port those to preview 2 without taking a wild guess at what semantics they need. The only known user of these functions was Enarx, who are no longer in business. If any other users need these sock functions to keep working in either the host or wasm adapter for preview 1, we'll need them to contribute to that support- all of the Bytecode Alliance contributors I have spoke to so far don't have a business case for supporting preview 1's sockets, and instead are all in on preview 2 sockets.
  3. After 2 releases in off-by-default legacy mode, we will delete the legacy preview 1 code from the repository. This means retiring the wasi-common, wasi-cap-std-sync, and wasi-tokio crates. We could publish a final empty version indicating they wont get any more updates and have been replaced with wasmtime-wasi.
@pchickey
Copy link
Contributor Author

I think this issue has lived its useful life and we can now call it closed - the preview 2 support is still not totally complete in wasmtime-wasi but it, and the specs themselves, are maturing rapidly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant