[pull] main from bytecodealliance:main #24
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
This commit adds a CI check that if versions are bumped that `cargo vet` still works. It's hoped that #9115 won't happen again in the future with this by ensuring we get all the various entries right in our vet configuration.
* Clean up dist build configuration * Move updating `$PATH` to the `main.js` script which is the one that mounts `/rust/bin` so that knowledge isn't spread around. * Remove some unused env vars in docker containers. * Forward cargo/rust-specific env vars to the build from outside of containers to the build itself. prtest:full * Change how musl rustflags are configured * More rustflags changes * Review feedback
Add missing support for $I128 in vec_elt_rev (which is a no-op). Fixes: #9090
Looking to unblock #9098
* Update nightly used in CI Move it up beyond the LLVM 19 upgrade to see if we have any issues with LLVM 19. prtest:full * Update nightly version * Fix some warnings on nightly * Alternative fix for warnings * More lint fixes * More warning tweaks
* Improve codegen for enums with many cases This commit improves the compile time of generating bindings for enums with many cases in them (e.g. 1000+). This is done by optimizing for enums specifically rather than handling them generically like other variants which can reduce the amount of code going into rustc to O(1) instead of O(N) with the number of cases. This in turn can greatly reduce compile time. The tradeoff made in this commit is that enums are now required to have `#[repr(...)]` annotations along with no Rust-level discriminants specified. This enables the use of a `transmute` to lift a discriminant into Rust with a simple bounds check. Previously this was one large `match` statement. Closes #9081 * Fix some tests * Add repr tag in fuzzing * Fix syntax for Rust 1.78
Previously we didn't pass the path to the Wasm file, so we didn't find the associated DWARF package.
* winch aarch64 wasm load * winch aarch64 wasm store * winch aarch64 fix float tests * winch aarch64 load tests & load_ptr * winch aarch64 store tests * winch aarch64 split ldr into uload & sload
* winch aarch64 extend * winch aarch64 wrap * winch aarch64 popcnt * winch aarch64 float promote & demote * winch aarch64 extend tests * winch aarch64 wrap tests * winch aarch64 popcnt tests * winch aarch64 promote & demote tests * winch ExtendKind methods * winch aarch64 popcnt: use scratch regiter * winch aarch64 popcnt: remove fallback tests * winch aarch64 popcnt: remove unused sizes cases * winch aarch64 tests: fix float loading * winch aarch64 popcnt: do not free scratch register
* Add the ability to generate async drop methods for resources.
In the component model, `resource.drop` is a canonical built-in without a proper name. So I invented a custom naming scheme for the component bindgen config. I went with:
`"[drop]{resource-name}"` where `{resource-name}` is the name as defined in WIT. e.g. `"[drop]input-stream"`.
This shouldn't conflict with anything existing in the wild as WIT identifiers are not allowed to contain square brackets.
* Add test for resource_async
This makes register allocation a bit faster when these logs are not used.
* ISLE: reduce allocations when lexing integers Instead of creating a temporary `Vec<u8>`, use a slice of the original underlying `buf`, and only allocate a temporary `String` if it contains an `_`. Copyright (c) 2024, Arm Limited. Signed-off-by: Karl Meakin <karl.meakin@arm.com> * ISLE: don't `vec![]` macro in lexer tests `Vec` can be compared against arrays, since both deref to slices. Copyright (c) 2024, Arm Limited. Signed-off-by: Karl Meakin <karlwfmeakin@gmail.com> * ISLE: create `Files` Centralize all file related arenas in `Files` struct. Copyright (c) 2024, Arm Limited. Signed-off-by: Karl Meakin <karl.meakin@arm.com> * ISLE: dont track line/col in `Pos` They are already tracked in `Files`, so no need to track them in `Pos` as well. This lets us simply the implementation of `Lexer::advance_pos` a bit. Copyright (c) 2024, Arm Limited. Signed-off-by: Karl Meakin <karl.meakin@arm.com> * ISLE: don't pass `Files` into every pass `Files` was being threaded through a lot of passes where it wasn't needed. It is only needed for reporting errors in `compile.rs` and for reporting line numbers when printing in `codegen.rs`. Copyright (c) 2024, Arm Limited. Signed-off-by: Karl Meakin <karl.meakin@arm.com> * ISLE: store `&str` in `Lexer` Store the text being lexed as `&str`, rather than `&[u8]`, so that substrings don't need to be rechecked for UTF-8 validity when lexing identifiers or integers. Copyright (c) 2024, Arm Limited. Signed-off-by: Karl Meakin <karl.meakin@arm.com> * ISLE: add `peek_byte` helper for lexer Copyright (c) 2024, Arm Limited. Signed-off-by: Karl Meakin <karl.meakin@arm.com> * ISLE: tests for lexing integers Copyright (c) 2024, Arm Limited. Signed-off-by: Karl Meakin <karl.meakin@arm.com> * ISLE: dont parse integers twice Instead of trying to parse an integer as an `i128`, and then as an `u128` if that fails, parse it only as a `u128` and then check for `i128::MIN`. Copyright (c) 2024, Arm Limited. Signed-off-by: Karl Meakin <karl.meakin@arm.com> --------- Signed-off-by: Karl Meakin <karl.meakin@arm.com> Signed-off-by: Karl Meakin <karlwfmeakin@gmail.com>
* Refactor use of `CodeBuilder` on the CLI This commit refactors `wasmtime run` and `wasmtime compile` to unconditionally use `CodeBuilder` internally. This will in theory help out in the future if more debug-related options are added to `CodeBuilder` for example. This refactoring required some changes to `CodeBuilder` to be able to support a query about whether the internal bytes were a component or a module. The text format is now converted to binary immediately when supplied rather than during the compilation phase. This in turn required some API changes to make the selection of supporting the text format a compile-time choice of method rather than a runtime value. * Fix compile * Fix no-cranelift build of CLI
* Pulley: Add memory access instructions with 64-bit offsets I had trimmed these instructions from the original upstreaming of the Pulley interpreter because I had mistakenly believed that they were unused. Turns out they are needed for Cranelift's Pulley backend to allow for lowering certain address modes to a single instruction. The alternative, lowering the address modes to a sequence of instructions, would be a bit annoying and these instructions seem generally useful. * rebase on top of indexing changes for `MachineState`
Looks like CMake before 3.20.0 doesn't generate newlines at all without this configuration option. CMake 3.20.0 and prior, however, generates newlines by default which is why this didn't show up in CI or development. Closes #9126
* Cranelift: Add a new backend for emitting Pulley bytecode This commit adds two new backends for Cranelift that emits 32- and 64-bit Pulley bytecode. The backends are both actually the same, with a common implementation living in `cranelift/codegen/src/isa/pulley_shared`. Each backend configures an ISA flag that determines the pointer size, and lowering inspects this flag's value when lowering memory accesses. To avoid multiple ISLE compilation units, and to avoid compiling duplicate copies of Pulley's generated `MInst`, I couldn't use `MInst` as the `MachInst` implementation directly. Instead, there is an `InstAndKind` type that is a newtype over the generated `MInst` but which also carries a phantom type parameter that implements the `PulleyTargetKind` trait. There are two implementations of this trait, a 32- and 64-bit version. This is necessary because there are various static trait methods for the mach backend which we must implement, and which return the pointer width, but don't have access to any `self`. Therefore, we are forced to monomorphize some amount of code. This type parameter is fairly infectious, and all the "big" backend types (`PulleyBackend<P>`, `PulleyABICallSite<P>`, etc...) are parameterized over it. Nonetheless, not everything is parameterized over a `PulleyTargetKind`, and we manage to avoid duplicate `MInst` definitions and lowering code. Note that many methods are still stubbed out with `todo!`s. It is expected that we will fill in those implementations as the work on Pulley progresses. * Trust the `pulley-interpreter` crate, as it is part of our workspace * fix some clippy warnings * Fix a dead-code warning from inside generated code * Use a helper for emitting br_if+comparison instructions * Add a helper for converting `Reg` to `pulley_interpreter::XReg` * Add version to pulley workspace dependency * search the pulley directory for crates in the publish script
The DWARF sections in `DebugInputContext` were always set to the DWARF sections from the .wasm file. However, when parsing split DWARF, most of the time we want to be using the sections from the .dwp file instead. We were already passing `gimli::Dwarf` to most places already so having the sections in `DebugInputContext` was no benefit. So a lot of this commit is replacing `context` with `dwarf`. Next, sometimes we were using the wrong `gimli::Dwarf`. In general, we want to use the skeleton unit for parsing line info, and the split unit for everything else. `clone_unit` was wrongly using the DWARF associated with the skeleton unit in many places. Instead of changing all of those places, I've kept the variable names `dwarf` and `unit`, but changed which DWARF they refer to. This also fits better with the non-split operation. So some of the changes in this commit are updating places that were already correct to use the new variable meanings. Finally, this commit adds a call to `copy_relocated_attributes`. This copies some info from the skeleton unit that is needed when parsing the split unit.
) * Add the ability to generate async drop methods for resources. In the component model, `resource.drop` is a canonical built-in without a proper name. So I invented a custom naming scheme for the component bindgen config. I went with: `"[drop]{resource-name}"` where `{resource-name}` is the name as defined in WIT. e.g. `"[drop]input-stream"`. This shouldn't conflict with anything existing in the wild as WIT identifiers are not allowed to contain square brackets. * Make `input-stream::drop` & `output-stream::drop` async * Prevent the guest from (inadvertently) spawning an unlimited amount of background tasks through the FileOutputStream * Properly clean up background tasks for even more types of I/O streams. Unlike FileOutputStream, the background tasks of these stream types are truly async. So aborting _without_ awaiting them was probably already good enough in practice. Nonetheless, waiting for the background to actually shut down just seems like good resource management "hygiene" to me. * Let HostInput/OutputStream provide specialized blocking_* implementations. * Rename filesystem's `spawn_blocking` -> `run_blocking` * Implement specialized `blocking_write_and_flush` for FileOutputStream. - `write` now always spawns the syscall on a background task, regardless of `allow_blocking_current_thread`. - `blocking_write_and_flush` is newly overridden and continues to do the `allow_blocking_current_thread` trickery that `write` used to do. * Implement `HostInputStream` for `FileInputStream`. - `read` always spawns the syscall on a background task, regardless of `allow_blocking_current_thread`. - `blocking_read` performs the `run_blocking`/`allow_blocking_current_thread` trickery. * In Preview1 adapter: ignore BlockingMode and always perform blocking I/O, as that's what preview1 did. * Remove special case for FileInputStream and change InputStream enum to be a type alias, just like OutputStream * Remove `[method]output-stream.forward` from bindgen config. It does not exist. * Refactor `blocking_splice` to take advantage of specialized `blocking_read` & `blocking_write_and_flush` implementations * Defer to regular `read` from within `blocking_read` to reduce duplication of logic.
This includes GNU split support for DWARF version 4.
* Implement a few minor optimizations around 128-bit integers This commit implements a few minor changes for `i128` in both the egraph optimizations and lowerings for x64. The optimization pass will now transform `iconcat` into a `uextend` or `sextend` where appropriate. The x64 backend then pattern-matches this to produce slightly more optimal machine code. Additionally the x64 backend now handles memory/immediate operands a bit better when the argument to a 128-bit operation is an `iconcat`. * Update test expectations * Match iadd lowering rules for isub
This adds `pooling-table-elements` and `pooling-max-core-instance-size` options to the CLI, allowing the user to override the defaults. I found myself needing to override both of these settings when running a ASP.NET Core application built using the Native AOT feature of the .NET runtime. Signed-off-by: Joel Dice <joel.dice@fermyon.com>
…#9082) This moves Wasmtime over from the old, regalloc-based stack maps system to the new "user" stack maps system. Removing the old regalloc-based stack maps system is left for follow-up work.
* riscv64: Add basic zfh support * riscv64: Add `f16const` and `bitcast` support * riscv64: Add `select` tests * riscv64: Add f16 load/stores * riscv64: Add misc f16 tests * riscv64: `cargo fmt` * riscv64: Restrict `fli` to 32/64 bit instructions We haven't implemented the lookup tables for f16 * riscv64: Check scalar ISA flags when checking supported vector types * riscv64: Restrict bitcasts to 16bit floating point values from/to vectors
* Remove unneeded `cu_low_pc` parameter `unit.low_pc` is the same value. * Simplify `clone_line_program` We were parsing things that had already been parsed into `gimli::Unit`, such as `DW_AT_comp_dir`, `DW_AT_name`, and `DW_AT_stmt_list`. * Use `gimli::Dwarf` methods in more places
* Move `compute_use_states` to be able to test it Make it a free-function so it doesn't depend on the `I` type variable of `Lower` * Don't force `Multiple` on multi-result instructions This commit is a result of [discussion on Zulip][Zulip] and is attempting to fix an issue where some 128-bit instructions aren't fully benefitting from load sinking on x64. At a high level 128-bit addition isn't able to sink loads into instructions for halves of the 128-bit operation. At a lower level the reason for this is that currently all operands of a multiple-result instruction are considered multiply-used (as each result could be used) which prevents load sinking. Operations on 128-bit integers may be coupled with `isplit` afterwards which is a multi-result instruction. This then means that the `Multiple` state flows backwards to the 128-bit operation and all its operands, including whatever is necessary to produce the individual components of each 128-bit integer. The fix in this commit is to introduce the concept of a "root" instruction from the perspective of the calculation of `ValueUseState`. In other words `ValueUseState` is no longer an accurate picture of the function as a whole, but only the parts of the function rooted at a "root" instruction. This is currently defined as multi-result instructions meaning that `isplit` for example is a root instruction. This is coupled with documentation/changes to `get_value_as_source_or_const` to never allow looking through root instructions (or considering them a `UniqueUse`). This commit additionally updates some documentation in a few places and refactors some usage of `get_value_as_source_or_const` to use other helpers instead to reduce callers of `get_value_as_source_or_const` and who to audit when modifying this function. [Zulip]: https://bytecodealliance.zulipchat.com/#narrow/stream/217117-cranelift/topic/ValueUseState.3A.3AMultiple.20and.20multi-result.20instructions/near/462833578 * Review comments
* cranelift/aarch64: support fnmsub and fnmadd * fix
This commit codifies the process [documented here](https://github.com/bytecodealliance/rfcs/blob/main/accepted/vulnerability-response-runbook.md) in the Wasmtime repository as it relates to Wasmtime itself. There's also a few minor changes from recent advisories such as: * We'll no longer use the publish-the-changes-from-the-advisory feature from GitHub. That basically just doesn't work any more. * PRs will instead be manually created to have CI run on them to weed out any issues. * Details about preparing the `main` branch ahead of the release are interleaved with the rest of the runbook. The intention is to supplement the official runbook with Wasmtime-specific information and flesh out a few minor steps we're following that are "extra" here too.
…t linking and subtyping (#9446) * Enable some miscellaneous GC spec tests These weren't caught by our checks that `should_fail` tests do not pass, I think because they only contain modules that should validate, and not any actual assertions. * Require declared subtyping when linking module import/export functions We were previously incorrectly doing a (shallow) match on structure when we should have been doing a subtyping check.
* Update `decommit_batch_size` type to `usize` * Add missing pooling options to `OptimizeOptions` * Add new pooling options to configuration * Add `max_memory_protection_keys` option also rename `memory_protection_keys` to `pooling_memory_protection_keys` * Add `total_gc_heaps` option * Add default options in comments
Help trim version of wasm-tools crates.
* Update docs/stabilization of wasm proposals This commit updates our documentation and documented status of various WebAssembly proposals. The goal of this commit it to explicitly list all requirements for each WebAssembly proposal in a way such that enabling a feature requires actively checking this table and performing effort to fill in. This is intended to mitigate GHSA-q8hx-mm92-4wvg where it was found that we were accidentally not fuzzing tail calls but had enabled it by default. This shuffles around some documentation, modernizes a bit, and notably tries to list out a green checkbox or red X for the status of various proposals. Notes for "holes" in the "on by default matrix" are also added. * Add a link to the GC tracking issue * Tweak the list of supported targets Android is listed in Tier 3 now
…9448) * Implement subtype checking for `[return_]call_indirect` instructions When Wasm GC is enabled, the `[return_]call_indirect` instructions must do full subtyping checks, rather than simple strict equality type checks. This adds an additional branch and slow path to indirect calls, so we only emit code for this check when Wasm GC is enabled, even though it would otherwise be correct to always emit it (because the `is_subtype` check would always fail for non-equal types, since there is no subtyping before Wasm GC). * fix no-gc builds
This helps us keep track of their code quality, and the way that some of them are reusing our implementation of `ref.test`.
This commit adds a test to the `wasmtime-fuzzing` crate as a sanity-check that eventually a module is generated requiring all of the features that wasmtime supports. This is intended to be another double-check in the process of enabling a proposal in Wasmtime by ensuring that the feature is added to this list which then transitively requires that fuzzing eventually generates a module needing that feature. cc #9449
… from host APIs (#9454) * Add support for initializing/getting/setting `funcref`s in GC structs from host APIs We implemented support for `funcref`s in both Wasm and host APIs for arrays, but somehow only implemented support for `funcref`s in structs for compiled Wasm code, and mistakenly forgot about them for structs and host APIs (and things that use host APIs, such as const expressions). * fix clippy
Move everything into the same location about what features are enabled, which are disabled, and such. Also statically enumerate all the features that wasm-smith enables for us and acknowledge that here too.
* Change how env vars work in `differential` fuzzer This commit updates the processing of the `ALLOWED_*` environment variables to work differently than before. Previously the list of engines and module-generation-strategies were filtered based on `ALLOWED_*` environment variables but this meant that the meaning of a fuzz input changed depending on environment variables. This commit instead replaces the handling of these environment variables to ensure that the meaning of the fuzz input doesn't change depending on their values. Instead fuzz test cases are early-thrown-out if they request an engine that's disabled or a module-generation-strategy that's disabled. * Update fuzz/fuzz_targets/differential.rs Co-authored-by: Trevor Elliott <awesomelyawesome@gmail.com> --------- Co-authored-by: Trevor Elliott <awesomelyawesome@gmail.com>
* Fix differential fuzzing with multi-memory and pooling Fix an issue found during fuzzing where when multi-memory and the pooling allocator are enabled then if MPK is detected and found the total number of memories needs to be increased because stores belong in a single stripe. * Update wasmi used in fuzzing Pulls in differential fuzzing support for multi-memory
This commit fixes the handling of default return types in `wasmtime-test-macros`. Prior to this commit, functions returning `()` inserted a `()` after the return output, which resulted in a syntax error. This commit fixes this issue by quoting empty tokens when the return type is the default one.
Nothing major in this update, just a new test to ignore due to not supporting exceptions.
* Update Wasmtime's WASI documentation. Replace most of Wasmtime's original WASI documentation with links to other resources that are now available. This documentation was written when WASI was first released, and while many of the big ideas are still present in WASI, many of the details have changed. For example, [Jco] is now a much better way to run WASI programs in Browsers than the original Web polyfill, and WASI is now goverened by the WASI Subgroup rather than by Wasmtime itself, and so on. [Jco]: https://github.com/bytecodealliance/jco Fixes #9107. * Fix documentation link. * Fix another doc link.
* wasi-nn: update OpenVINO bindings Recent changes to OpenVINO's C bindings introduced breaking changes that prevent use of older OpenVINO libraries with newer bindings (see [openvino-rs#143] if you're interested in the details). This updates both the bindings and the version of OpenVINO used to test against. Fixes #9379. [openvino-rs#143]: intel/openvino-rs#143 * vet: certify openvino version updates All of these updates include code I've reviewed. * ci: update `install-openvino-action` prtest:full
This commit fixes some issues cropping up due to fuzzing 128-bit atomics
in `cranelift-{icache,fuzzgen}`. Not all backends support these
operations and x64 only supports them with a feature enabled, so for now
disable fuzzing until it's ready to fuzz later.
This commit fixes a timeout that was found by OSS-Fuzz recently where a module was calling `memory.grow` many times, presumably in a loop, with a modest amount each time. This meant that `memory.grow` was taking, in total, a quadratic amount of time because Wasmtime was configured with dynamic memories and no memory was reserved for growth. That in turn meant that the test case eventually timed out due to this quadratic behavior. To fix this in addition to the memory allocation cap that we already track a new cap for the number of times memories/tables can be grown was also added. Any growth beyond this limit is rejected and helps prevent this quadratic behavior.
* Add general fuzzing support for custom page sizes * Add custom-page-sizes as an expected feature for module generation
* Move JIT debug image registration to CodeMemory JIT images correspond to ELF images, which may represent multiple modules within a single component. * Add the last ifdef * Remove ManuallyDrop as per feedback
* Don't bounds-check GC refs returned from trusted libcalls * Refactor GC heap base/bound helper functions Allow their independent usage. Disas test updates are just mechanical value renumberings. * Mark loading the activations table pointer from the vmctx as `readonly` It is never modified, should produce better codegen when we repeatedly access it.
This commit adds a guard to `splat_const` lowering on AArch64 to ensure that unsupported sizes don't make their way to the backend to avoid a panic. The included test here currently panics on `main` when passed to `wasmtime compile` and succeeds afterwards.
…bcall (#9483) It is unnecessary for Wasm code to do this bounds check because the libcall is trusted. We already avoided it for arrays, but forgot to do structs as well.
* s390x: Hard code the link register for ElfTlsGetOffset The ElfTlsGetOffset pseudo-instruction already hard codes the rest of the abi anyway and if another abi is ever needed, adding another pseudo-instruction or an enum field to the existing one would likely be necessary anyway. * Remove a couple of dead_code allows
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )