Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

working with gc types #317

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Remove reference type options
  • Loading branch information
oovm committed Mar 20, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit 75d592501305f21d25db7e9f5837eed682c0a175
2 changes: 1 addition & 1 deletion design/mvp/Binary.md
Original file line number Diff line number Diff line change
@@ -276,7 +276,7 @@ canonopt ::= 0x00 => string-encod
| 0x03 m:<core:memidx> => (memory m)
| 0x04 f:<core:funcidx> => (realloc f)
| 0x05 f:<core:funcidx> => (post-return f)
| 0x06 vec(<ref-options>) => (reference-type opt*) 📡
| 0x06 => reference-type 📡
```
Notes:
* The second `0x00` byte in `canon` stands for the `func` sort and thus the
34 changes: 14 additions & 20 deletions design/mvp/Explainer.md
Original file line number Diff line number Diff line change
@@ -1144,7 +1144,7 @@ canonopt ::= string-encoding=utf8
| (memory <core:memidx>)
| (realloc <core:funcidx>)
| (post-return <core:funcidx>)
| (reference-type <ref-opts>*) 📡
| reference-type 📡
```
While the production `externdesc` accepts any `sort`, the validation rules
for `canon lift` would only allow the `func` sort. In the future, other sorts
@@ -1182,15 +1182,11 @@ validation requires this option to be present (there is no default).

##### `reference-type` options

📡 `(reference-type ...)` does not take effect by default. When this option
📡 `reference-type` does not take effect by default. When this option
is turned on, Canonical ABI will be required to use reference types to pass
parameters. This option conflicts with `memory` and `realloc` and cannot
exist at the same time.

📡 In the MVP version, the reference type refers to the specific monomorphic
type, such as `(ref array (mut u8))`, instead of type-erased
`(ref eq)`.

📡 When `reference-type` is enabled, the parameter type will change as follows:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the right place to put this spec information is in CanonicalABI.md (along-side the current spec information for how non-gc works) and in canonical-abi/definitions.py (which has the benefit that you can write tests for it in run_tests.py)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how to rewrite definitions.py. The binary needs to reference an index of an already defined type.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since definitions.py is not a complete Python reference implementation but, rather, just a "suggestive" subset that describes just the lifting/lowering/built-in rules, what we do is, for canon definitions that take a typeidx in the binary format, we have the corresponding canon_* Python function take a Python object that directly represents the type (e.g., see how canon_lift takes a FuncType directly). That being said, I don't think there are any cases where you'll need to take a core wasm gc type as an immediate -- core wasm types are programmatically derived from component-level types by, e.g., flatten_functype which is called by canon_lift and canon_lower. Thus, I think you just need to add CoreArrayType and CoreStructType Python classes (analogous to CoreFuncType) so they can be created by flatten_functype in the appropriate cases when cx.opts.gc is true (which you'd also add to CanonicalOptions.


| wit type | wasm w/o `reference-type` | wasm w/ `reference-type` |
@@ -1206,20 +1202,18 @@ type, such as `(ref array (mut u8))`, instead of type-erased
| `record { a, b }` | `(a: A, b: B)` (flatten) | `(ref struct $record,)` |
| `tuple<A, B>`(in⩽16) | `(a: A, b: B)` (flatten) | `(a: A, b: B)` (flatten) |
| `tuple<A, B>`(out>1) | `(ptr: i32)` | `(ref struct (mut $a) (mut $b),)` |
| `result<A, B>` | `(i32, i32)` | `(i32, (ref eq))` |
| `variant` | `(i32, [MAX_VARIANT_SIZE])` | `(i32, (ref eq))` |

📡 Nullable types are treated specially in many languages and are therefore not
considered variants when the `reference-types` option is on.

| wit type | wasm w/o `reference-type` | wasm w/ `reference-type` |
|:-----------------------|:--------------------------|:-------------------------|
| `option<bool>` | `(i32, i32)` | `(ref null i31)` |
| `option<char>` | `(i32, i32)` | `(ref null i32)` |
| `option<i8>` | `(i32, i32)` | `(ref null i31)` |
| `option<i32>` | `(i32, i32)` | `(ref null i32)` |
| `option<i64>` | `(i32, i64)` | `(ref null i64)` |
| `option<T>` (ref type) | `(i32, SIZE_OF_T)` | `(ref null $t)` |

📡 For variant types, they will be unpacked into two parts: enumeration and data.

| wit type | wasm w/o `reference-type` | wasm w/ `reference-type` |
| :---------------------- | :-------------------------- | :----------------------- |
| `option<A>` (heap type) | `(i32, [A_SIZE])` | `(i32, (ref null eq))` |
| `none` (heap type) | `(0, [FILL_ZEROES])` | `(1, null)` |
| `(some A)` (heap type) | `(1, [A_SIZE])` | `(0, ref $a)` |
| `result<A, B>` | `(i32, [MAX_VARIANT_SIZE])` | `(i32, (ref null eq))` |
| `(ok A)` | `(0, [A_SIZE])` | `(0, ref $a)` |
| `(err B)` | `(1, [B_SIZE])` | `(1, ref $b)` |
| `variant` | `(i32, [MAX_VARIANT_SIZE])` | `(i32, (ref null eq))` |

##### `post-return` options

8 changes: 0 additions & 8 deletions design/mvp/FutureFeatures.md
Original file line number Diff line number Diff line change
@@ -47,14 +47,6 @@ version, which may lead to unexpected semantic damage. This can only be
guaranteed by the language compiler for the time being, and it is not
cross-language safe.

## Integrate `stringref` with reference types 📡

The current wit `string` type is regarded as `(array (u8))` or `(array (mut u8))`.
When the `stringref` proposal is stable, it can be considered to be mapped to
`stringref`. In other words, the `reference-type` option can work together with
the `string-encoding` option.


## Shared-some-things linking via "adapter modules"

The original [Interface Types proposal] and the re-layered [Module Linking
132 changes: 77 additions & 55 deletions design/mvp/canonical-abi/definitions.py
Original file line number Diff line number Diff line change
@@ -300,6 +300,7 @@ class CanonicalOptions:
string_encoding: str
realloc: Callable[[int,int,int,int],int]
post_return: Callable[[],None]
reference_type: bool

class ComponentInstance:
may_leave: bool
@@ -941,44 +942,56 @@ def lift_flat_signed(vi, core_width, t_width):
return i

def lift_flat_string(cx, vi):
ptr = vi.next('i32')
packed_length = vi.next('i32')
return load_string_from_range(cx, ptr, packed_length)
if cx.opts.reference_type:
raise NotImplementedError
else:
ptr = vi.next('i32')
packed_length = vi.next('i32')
return load_string_from_range(cx, ptr, packed_length)

def lift_flat_list(cx, vi, elem_type):
ptr = vi.next('i32')
length = vi.next('i32')
return load_list_from_range(cx, ptr, length, elem_type)
if cx.opts.reference_type:
raise NotImplementedError
else:
ptr = vi.next('i32')
length = vi.next('i32')
return load_list_from_range(cx, ptr, length, elem_type)

def lift_flat_record(cx, vi, fields):
record = {}
for f in fields:
record[f.label] = lift_flat(cx, vi, f.t)
return record
if cx.opts.reference_type:
raise NotImplementedError
else:
record = {}
for f in fields:
record[f.label] = lift_flat(cx, vi, f.t)
return record

def lift_flat_variant(cx, vi, cases):
flat_types = flatten_variant(cases)
assert(flat_types.pop(0) == 'i32')
case_index = vi.next('i32')
trap_if(case_index >= len(cases))
class CoerceValueIter:
def next(self, want):
have = flat_types.pop(0)
x = vi.next(have)
match (have, want):
case ('i32', 'f32') : return decode_i32_as_float(x)
case ('i64', 'i32') : return wrap_i64_to_i32(x)
case ('i64', 'f32') : return decode_i32_as_float(wrap_i64_to_i32(x))
case ('i64', 'f64') : return decode_i64_as_float(x)
case _ : return x
c = cases[case_index]
if c.t is None:
v = None
if cx.opts.reference_type:
raise NotImplementedError
else:
v = lift_flat(cx, CoerceValueIter(), c.t)
for have in flat_types:
_ = vi.next(have)
return { case_label_with_refinements(c, cases): v }
flat_types = flatten_variant(cases)
assert(flat_types.pop(0) == 'i32')
case_index = vi.next('i32')
trap_if(case_index >= len(cases))
class CoerceValueIter:
def next(self, want):
have = flat_types.pop(0)
x = vi.next(have)
match (have, want):
case ('i32', 'f32') : return decode_i32_as_float(x)
case ('i64', 'i32') : return wrap_i64_to_i32(x)
case ('i64', 'f32') : return decode_i32_as_float(wrap_i64_to_i32(x))
case ('i64', 'f64') : return decode_i64_as_float(x)
case _ : return x
c = cases[case_index]
if c.t is None:
v = None
else:
v = lift_flat(cx, CoerceValueIter(), c.t)
for have in flat_types:
_ = vi.next(have)
return { case_label_with_refinements(c, cases): v }

def wrap_i64_to_i32(i):
assert(0 <= i < (1 << 64))
@@ -1026,35 +1039,44 @@ def lower_flat_string(cx, v):
return [Value('i32', ptr), Value('i32', packed_length)]

def lower_flat_list(cx, v, elem_type):
(ptr, length) = store_list_into_range(cx, v, elem_type)
return [Value('i32', ptr), Value('i32', length)]
if cx.opts.reference_type:
raise NotImplementedError
else:
(ptr, length) = store_list_into_range(cx, v, elem_type)
return [Value('i32', ptr), Value('i32', length)]

def lower_flat_record(cx, v, fields):
flat = []
for f in fields:
flat += lower_flat(cx, v[f.label], f.t)
return flat
if cx.opts.reference_type:
raise NotImplementedError
else:
flat = []
for f in fields:
flat += lower_flat(cx, v[f.label], f.t)
return flat

def lower_flat_variant(cx, v, cases):
case_index, case_value = match_case(v, cases)
flat_types = flatten_variant(cases)
assert(flat_types.pop(0) == 'i32')
c = cases[case_index]
if c.t is None:
payload = []
if cx.opts.reference_type:
raise NotImplementedError
else:
payload = lower_flat(cx, case_value, c.t)
for i,have in enumerate(payload):
want = flat_types.pop(0)
match (have.t, want):
case ('f32', 'i32') : payload[i] = Value('i32', encode_float_as_i32(have.v))
case ('i32', 'i64') : payload[i] = Value('i64', have.v)
case ('f32', 'i64') : payload[i] = Value('i64', encode_float_as_i32(have.v))
case ('f64', 'i64') : payload[i] = Value('i64', encode_float_as_i64(have.v))
case _ : pass
for want in flat_types:
payload.append(Value(want, 0))
return [Value('i32', case_index)] + payload
case_index, case_value = match_case(v, cases)
flat_types = flatten_variant(cases)
assert(flat_types.pop(0) == 'i32')
c = cases[case_index]
if c.t is None:
payload = []
else:
payload = lower_flat(cx, case_value, c.t)
for i,have in enumerate(payload):
want = flat_types.pop(0)
match (have.t, want):
case ('f32', 'i32') : payload[i] = Value('i32', encode_float_as_i32(have.v))
case ('i32', 'i64') : payload[i] = Value('i64', have.v)
case ('f32', 'i64') : payload[i] = Value('i64', encode_float_as_i32(have.v))
case ('f64', 'i64') : payload[i] = Value('i64', encode_float_as_i64(have.v))
case _ : pass
for want in flat_types:
payload.append(Value(want, 0))
return [Value('i32', case_index)] + payload

def lower_flat_flags(v, labels):
i = pack_flags_into_int(v, labels)
11 changes: 6 additions & 5 deletions design/mvp/canonical-abi/run_tests.py
Original file line number Diff line number Diff line change
@@ -32,16 +32,17 @@ def realloc(self, original_ptr, original_size, alignment, new_size):
self.memory[ret : ret + original_size] = self.memory[original_ptr : original_ptr + original_size]
return ret

def mk_opts(memory = bytearray(), encoding = 'utf8', realloc = None, post_return = None):
def mk_opts(memory = bytearray(), encoding = 'utf8', realloc = None, post_return = None, reference_type = False):
opts = CanonicalOptions()
opts.memory = memory
opts.string_encoding = encoding
opts.realloc = realloc
opts.post_return = post_return
opts.reference_type = reference_type
return opts

def mk_cx(memory = bytearray(), encoding = 'utf8', realloc = None, post_return = None):
opts = mk_opts(memory, encoding, realloc, post_return)
def mk_cx(memory = bytearray(), encoding = 'utf8', realloc = None, post_return = None, reference_type = False):
opts = mk_opts(memory, encoding, realloc, post_return, reference_type)
return CallContext(opts, ComponentInstance())

def mk_str(s):
@@ -234,9 +235,9 @@ def test_string(src_encoding, dst_encoding, s):
for s in fun_strings:
test_string(src_encoding, dst_encoding, s)

def test_heap(t, expect, args, byte_array):
def test_heap(t, expect, args, byte_array, reference_type=False):
heap = Heap(byte_array)
cx = mk_cx(heap.memory)
cx = mk_cx(heap.memory, reference_type=reference_type)
test(t, args, expect, cx)

# Empty record types are not permitted yet.