Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wasmtime: Implement table.get and table.set #1923

Merged
merged 3 commits into from
Jun 30, 2020

Conversation

fitzgen
Copy link
Member

@fitzgen fitzgen commented Jun 25, 2020

These instructions have fast, inline JIT paths for the common cases, and only
call out to host VM functions for the slow paths. This required some changes to
cranelift-wasm's FuncEnvironment: instead of taking a FuncCursor to insert
an instruction sequence within the current basic block,
FuncEnvironment::translate_table_{get,set} now take a &mut FunctionBuilder
so that they can create whole new basic blocks. This is necessary for
implementing GC read/write barriers that involve branching (e.g. checking for
null, or whether a store buffer is at capacity).

Furthermore, it required that the load, load_complex, and store
instructions handle loading and storing through an r{32,64} rather than just
i{32,64} addresses. This involved making r{32,64} types acceptable
instantiations of the iAddr type variable, plus a few new instruction
encodings.

Part of #929

@fitzgen fitzgen requested a review from sunfishcode June 25, 2020 18:42
@alexcrichton alexcrichton changed the base branch from master to main June 25, 2020 18:48
@github-actions github-actions bot added cranelift Issues related to the Cranelift code generator cranelift:meta Everything related to the meta-language. cranelift:wasm wasi Issues pertaining to WASI labels Jun 25, 2020
@github-actions
Copy link

Subscribe to Label Action

cc @bnjbvr, @kubkon

This issue or pull request has been labeled: "cranelift", "cranelift:meta", "cranelift:wasm", "wasi"

Thus the following users have been cc'd because of the following labels:

  • bnjbvr: cranelift
  • kubkon: wasi

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Could this add some more tests too? Some things like:

  • Using table.get/table.set on a funcref table
  • Calling table.set with a ref.null
  • Calling table.get returning a ref.null
  • calling table.set where the previous element is null
  • Calling table.set where the previous element had a refcount of 1 (this may be tricky?)

crates/environ/src/func_environ.rs Outdated Show resolved Hide resolved
crates/runtime/src/libcalls.rs Outdated Show resolved Hide resolved
@fitzgen
Copy link
Member Author

fitzgen commented Jun 25, 2020

Could this add some more tests too? Some things like:

All of these cases are in the spec tests (except explicitly targeting replacing a ref_count == 1 element), and enabled in this PR. I don't think it makes sense to duplicate them (again, excpet for ref_count == 1).

@alexcrichton
Copy link
Member

Hm so I do think that it's worthwhile to add some more tests here. I'm a bit hesitant to rely on pure wast tests since all the interesting bits are about how host code interacts with this.

For example this program will hit "attempt to subtract with overflow" because the reference count is decremented in wasm but then drop_in_place tries to decrement it again. If that's fixed then the program segfaults since it tries to drop A twice. The problem is that we're in the process of dropping a slot in the table when at the same time we still have access to read that slot. What needs to happen is we need to move out of the table, move the new entry into the table, then drop the previous entry. That way if in the destructor of the previous value we read the table slot we get the new value.

I do realize that the spec tests try to be somewhat exhaustive, but given that this was pretty easy to segfault I think it's worthwhile trying to be at least somewhat flavorful with tests that exercise the embedding where we have more control over GC and such.

@fitzgen
Copy link
Member Author

fitzgen commented Jun 25, 2020

I do realize that the spec tests try to be somewhat exhaustive, but given that this was pretty easy to segfault I think it's worthwhile trying to be at least somewhat flavorful with tests that exercise the embedding where we have more control over GC and such.

Yeah I aleady had a fix for the drop_in_place bit, but the destructor-touching-the-table bit needs some tests for sure. I have a half written one already.

I don't think we need to duplicate anything that's already in the spec tests, just the ref_count == 1 paths, and dtors that touch the table.

@fitzgen fitzgen force-pushed the table-get-and-table-set branch 2 times, most recently from 13d3537 to 7499f7b Compare June 26, 2020 19:51
@fitzgen
Copy link
Member Author

fitzgen commented Jun 26, 2020

Okay, I've added a bunch of tests for pathological gc-in-externref-destructor-while-sweeping behaviors and other various re-entry into GC.

Working on a fuzz target specifically for exercising table.get, table.set, and GC now, but it will probably be a follow up PR.

Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh man gc-in-dtor is pretty gnarly, totally forgot to think about that! Very nice tests though 👍

This all looks great to me, but as you mentioned at the beginning someone should take a look at the cranelift pieces too.

crates/environ/src/func_environ.rs Outdated Show resolved Hide resolved
crates/runtime/src/libcalls.rs Outdated Show resolved Hide resolved
crates/runtime/src/externref.rs Show resolved Hide resolved
crates/runtime/src/externref.rs Show resolved Hide resolved
@fitzgen fitzgen force-pushed the table-get-and-table-set branch from 7499f7b to 725e713 Compare June 26, 2020 20:19
@fitzgen
Copy link
Member Author

fitzgen commented Jun 26, 2020

Alright, I got the fuzz target written and passing as well!

@github-actions github-actions bot added fuzzing Issues related to our fuzzing infrastructure wasmtime:api Related to the API of the `wasmtime` crate itself labels Jun 26, 2020
@github-actions
Copy link

Subscribe to Label Action

cc @fitzgen, @peterhuene

This issue or pull request has been labeled: "fuzzing", "wasmtime:api"

Thus the following users have been cc'd because of the following labels:

  • fitzgen: fuzzing
  • peterhuene: wasmtime:api

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

Copy link
Member

@sunfishcode sunfishcode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The meta and frontend parts look good to me, with one minor refactoring request:

cranelift/frontend/src/frontend.rs Show resolved Hide resolved
fitzgen added 3 commits June 30, 2020 12:00
These instructions have fast, inline JIT paths for the common cases, and only
call out to host VM functions for the slow paths. This required some changes to
`cranelift-wasm`'s `FuncEnvironment`: instead of taking a `FuncCursor` to insert
an instruction sequence within the current basic block,
`FuncEnvironment::translate_table_{get,set}` now take a `&mut FunctionBuilder`
so that they can create whole new basic blocks. This is necessary for
implementing GC read/write barriers that involve branching (e.g. checking for
null, or whether a store buffer is at capacity).

Furthermore, it required that the `load`, `load_complex`, and `store`
instructions handle loading and storing through an `r{32,64}` rather than just
`i{32,64}` addresses. This involved making `r{32,64}` types acceptable
instantiations of the `iAddr` type variable, plus a few new instruction
encodings.

Part of bytecodealliance#929
This new fuzz target exercises sequences of `table.get`s, `table.set`s, and
GCs.

It already found a couple bugs:

* Some leaks due to ref count cycles between stores and host-defined functions
  closing over those stores.

* If there are no live references for a PC, Cranelift can avoid emiting an
  associated stack map. This was running afoul of a debug assertion.
@fitzgen fitzgen force-pushed the table-get-and-table-set branch from 95ebe78 to a2f4202 Compare June 30, 2020 19:02
@fitzgen fitzgen merged commit 3ddd024 into bytecodealliance:main Jun 30, 2020
@fitzgen fitzgen deleted the table-get-and-table-set branch June 30, 2020 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cranelift:meta Everything related to the meta-language. cranelift:wasm cranelift Issues related to the Cranelift code generator fuzzing Issues related to our fuzzing infrastructure wasi Issues pertaining to WASI wasmtime:api Related to the API of the `wasmtime` crate itself
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants