Inert "raw pointer" for JsValue

[RSSchermer]

I'm working on an abstraction around the WebGl2/OpenGL ES3 interface that I'm hoping to make thread-/worker-safe. The core concept revolves around secondary threads/workers constructing monadic Task objects which are then submitted back to the main thread (or hopefully, one day, an offscreen-canvas thread/worker). For the ergonomics of the API it's important that certain resource handles (buffers, textures, programs, etc.) can live in these secondary threads, and this is where I run into problems.

These resource handles need to be associated with the actual OpenGL GPU resources they represent. In OpenGL I can do this by storing the actual GLUints in a resource handle. Of course, the web platform chose to integrate the deletion of GPU resources with JS garbage collection by wrapping them in JS objects, which makes this not an option for WebGl.

JsValue is - understandably - not Send/Sync, which means I also cannot store the JsValue in my resource handles. With the current public wasm-bindgen interface, the only option this seems to leave is to maintain a hashtable on the main thread which maps my own identifiers to JsValues. That would however kind of duplicate wasm-bindgen's own internal hashtable lookup, which I'd rather avoid.

Given wasm-bindgen's current implementation, it seems that perhaps the idx stored inside a JsValue would be the ideal way to associate my resource handles with the WebGl objects. This left me wondering if perhaps an analogue to Box's into_raw and from_raw would be possible/desirable:

#[derive(Clone, Copy)]
struct JsId {
    idx: u32
}

impl JsValue {
    pub fn into_raw(js_value: JsValue) -> JsId {
        let idx = js_value.idx;
        
        mem::forget(js_value);

        JsId {
            idx
        }
    }

    pub unsafe fn from_raw(raw: JsId) {
        JsValue {
            idx: raw.idx,
            _marker: marker::PhantomData,
        }
    }
}

The caller of into_raw would then be responsible for freeing the JsValue entry in wasm-bindgen's hashtable by calling from_raw at a later time. The documentation for from_raw would mention that it is only safe to call it on a JsId in the same thread/worker that originally created the JsId. The JsId would be otherwise inert, it simply serves as a thread-safe identifier until its converted back into a JsValue.

Would something like this be considered? Also, would there be an alternative after the host-bindings proposal lands? Would JsValue's idx then be replaced with some sort of anyref as mentioned in the reference-types proposal? I've looked at the host-bindings, threads and reference-types proposals, but I can't quite evaluate whether or not there will be a mechanism that would prevent these anyrefs from being shared/passed between threads/workers. Perhaps someone here is more intimately involved with the inner workings of current and future WASM and knows this?

[alexcrichton]

Thanks for the report, this is definitely an interesting idea!

It's definitely possible that we could do something like this today, but I'm also not entirely sure if we want to. I believe this is specifically empowered by the representation strategy of JsValue today and such an efficient implementation wouldn't be possible in an anyref-enabled world.

That being said maybe it's not a bad idea to expose what we have today? It's not clear if we'll be able to switch to a different representation any time soon, in which case providing the raw accessors may make sense.

[RSSchermer]

It would certainly help me, but I'd also understand not wanting to set up wasm-bindgen for a breaking change (although after studying the WASM proposals I suspect there's a good chance that taking full advantage of these might break the current API anyway?). From my perspective, the worst case post-anyref scenario would be having to switch to a custom hashtable anyway, but if wasm-bindgen's lookup would be eliminated by anyref that wouldn't really be a performance degradation. I'm still holding out some hope though that there'll be a better alternative post-anyref.

I'd be happy to make a PR.

[Pauan]

or hopefully, one day, an offscreen-canvas thread/worker

This is already possible today:

https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas

You can render to an offscreen canvas in a worker and then send the bitmap to the main thread (transferring the bitmap doesn't copy anything, so it's very fast).

Perhaps someone here is more intimately involved with the inner workings of current and future WASM and knows this?

I'm not an expert, but my understanding is that threads can only share linear memory, they cannot share tables, and anyref always uses tables (or the stack). So as far as I know, anyref cannot be sent between threads.

However, I don't think all is lost: it is possible to use the shared linear memory to synchronize between threads.

The basic idea would be something like this:

1. Thread A stores the anyref in a table.

2. Thread A sends the table index (which is just an i32) to thread B (probably using synchronization).

3. Whenever thread B wants to do an operation on the index, it sends the index back to thread A (using synchronization), and thread A then looks up the index in the table and does the operation.

4. When thread B is done with the index, it sends a message to thread A asking thread A to remove the index from the table.

We still have a lot of work to do to make all this synchronization stuff work, but I think it's possible, at least in theory.

Note: these tables are wasm tables, not JS tables. They're not hash tables, they're more like Vec in Rust. So lookups are very fast.

---

As for adding new methods to JsValue: since JsValue will eventually be replaced with anyref, I don't think adding this to JsValue is a good idea. It would mean that all code would pay a performance penalty, even if it doesn't need to send JsValue between threads.

However! I think it's reasonable to provide this functionality with a new type, perhaps called JsValueSync or similar.

[RSSchermer]

Thanks @Pauan!

Yes! The offscreen canvas API is exciting! Unfortunately, browser support doesn't seem great yet, but hopefully the day comes sooner rather than later.

You may well be right about the shareabilty of anyref. I've taken another look at the GC proposal overview; it might imply an intend to subtype anyref at some point in the future to indicate whether or not it can be stored in shared/linear memory on a case by case basis (ctrl+f "threads", last entry), but I'm not sure at all (I have a feeling that all host-binding related anyrefs would be marked non-shareable anyway).

What you outline is pretty much exactly my current plan:

1. Secondary thread creates a new resource handle, say for a WebGl buffer, that is uninitialized: contains something like an Arc<Option<JsId>> (even though I'd be writing to it later, no Mutex would be necessary I think, the design already guarantees that there won't be concurrent access) that is initialized to None, to be lazily initialized later.
2. Secondary thread creates a GPU Task that uses the buffer handle for the first time (say a Task that uploads data to the buffer), submits the task to the "executor thread" via an mpsc channel/message queue.
3. The executor processes the Task, initializes the buffer handle by creating the actual WebGlBuffer object, converting it to a JsId, and updating the Arc.
4. Any subsequent Task that uses the buffer handle, temporarily reconstructs the WebGlBuffer object from the JsId and then mem::forgets the WebGlBuffer again when it's done with it.
5. When the buffer handle eventually gets dropped, submit a special Task to the executor that reconstructs the WebGlBuffer again for one final time without mem::forgetting it to clean up the JsValue entry (and maybe also calls delete_buffer on the WebGl context, I'm not yet sure if that performs better than leaving it to the JS garbage collector).

Your outline also seems pretty analogous to the current wasm-bindgen situation, except rather than a thread-local wasm-bindgen hashtable, there'd be a thread-local WASM table, and rather than a key into the wasm-bindgen hashtable, JsValue would hold an index into the WASM table? The reference types proposal overview also mentions that anyrefs could be stored in "global memory", although I'm not at all familiar enough with WASM terminology to understand what "global memory" means in this case ("global memory" != "WASM module linear memory"?).

As for adding new methods to JsValue: since JsValue will eventually be replaced with anyref, I don't think adding this to JsValue is a good idea. It would mean that all code would pay a performance penalty, even if it doesn't need to send JsValue between threads.

However! I think it's reasonable to provide this functionality with a new type, perhaps called JsValueSync or similar.

I'm not sure I completely understand. I don't think into_raw and from_raw should be considered methods exactly, they're more like associated functions perhaps (they don't take self)? Of course you could kind of invert them and associate them with JsId (JsValueSync?) instead (JsId::from_js_value, JsId::into_js_value), I'm not sure what would be a better semantic fit. I don't quite understand why that would have performance implications for code that does not use this feature, at least not with the current wasm-bindgen implementation.

[Pauan]

Your outline also seems pretty analogous to the current wasm-bindgen situation, except rather than a thread-local wasm-bindgen hashtable, there'd be a thread-local WASM table, and rather than a key into the wasm-bindgen hashtable, JsValue would hold an index into the WASM table?

Basically, yeah, except it would have to be thread-aware (with synchronization), whereas the current implementation is not.

The reference types proposal overview also mentions that anyrefs could be stored in "global memory"

I don't see any mention of "global memory". There are mentions of globals, but those are basically just thread-local variables.

(Technically they're instance-local, not thread-local, but in practice it will be one instance per thread, so it's the same thing.)

In any case, globals are completely different from linear memory, and globals are not shared between threads. Only linear memory is shared.

I don't quite understand why that would have performance implications for code that does not use this feature, at least not with the current wasm-bindgen implementation.

Sure, there's no performance cost with the current implementation.

But it looks like anyref might be landing pretty soon: it's currently in phase 3, and Firefox already ships experimental support for it (behind a flag).

Right now, wasm-bindgen stores JS values in a slab. This is necessary because of the lack of anyref.

When anyref becomes widely implemented, we will be switching JsValue to use anyref. And then we won't need to store things in a slab anymore, since we can just use anyref directly. So that gives a nice performance boost.

But with your proposal, we would still need to store things in a slab, even with anyref. So that's a performance cost.

That's why I suggested a new type: single-threaded usage will continue to use JsValue (which will use anyref, and is not Send or Sync), and a new JsValueSync type could be added (which would use the wasm table lookup strategy, and can be sent across threads). That way you only pay for what you need.

[RSSchermer]

In any case, globals are completely different from linear memory, and globals are not shared between threads. Only linear memory is shared.

Ah, that makes sense!

When anyref becomes widely implemented, we will be switching JsValue to use anyref. And then we won't need to store things in a slab anymore, since we can just use anyref directly. So that gives a nice performance boost.

But with your proposal, we would still need to store things in a slab, even with anyref. So that's a performance cost.

I guess my confusion is around what JsValue would be a type wrapper for in the anyref world in terms of WASM primitives. Would it become a typewrapper for some an anyref primitive? I can't quite wrap my head around how that would work if anyrefs can't be stored in linear memory (an actual anyref primitive can only ephemerally live "on the stack" after you obtain it with table.get or by calling an anyfunc that returns one? They can't leave the stack frame in which they are created other than via table.set?). Would JsValue then hold some sort of (i32 table-identifier, i32 element-index) pair? I guess I kind of assumed it would still end up having to wrap some sort of numeric type so that it can be passed up and down the stack, stored in a Box, cloned, etc. It seems that in that case it ought to work? Like I said though, my knowledge on this is severely lacking and my intuition around this is very poor, I might be completely off base.

The other thing I don't quite understand yet is: if JsValueSync is to be something completely separate from JsValue, how would I obtain one? Presumably if I called say WebGl2RenderingContext::create_buffer, it would still have to return a (typed wrapper around a) JsValue.

[Pauan]

I guess my confusion is around what JsValue would be a type wrapper for in the anyref world in terms of WASM primitives. Would it become a typewrapper for some an anyref primitive?

That's a good question! I think @alexcrichton can probably answer that better. But I imagine it would be a magical struct which is translated by wasm-bindgen into anyref. Similar to how some of wasm-bindgen's functions are magical and get translated into raw wasm operations.

an actual anyref primitive can only ephemerally live "on the stack" after you obtain it with table.get or by calling an anyfunc that returns one?

Yes, or by calling an imported function which returns an anyref (or a wasm function which returns an anyref). Calling imported functions will be the primary way to obtain an anyref.

They can't leave the stack frame in which they are created other than via table.set?

anyref is a first-class value: wasm functions can accept an anyref as an argument, they can store anyref in local variables, and they can also return an anyref

So tables are only needed if you want the anyref to be heap-allocated.

Though I know some languages (like C and C++, maybe also Rust?) use a shadow stack in linear memory to make pointers work, so perhaps there will need to be a similar "shadow table" for anyref, which means that sometimes stack-allocated anyref will still be put into a table.

To be clear, the linear memory is primarily for the heap (and a few other things, but mostly the heap). The stack is automatically managed by the wasm engine, it's separate from the linear memory.

In other words, the linear memory is for heap allocated bytes, tables are for heap allocated anyref (and other things), and the stack is separate from both of them.

The linear memory is just a chunk of bytes, so if it was possible to store anyref in linear memory, then wasm programs could access the numeric pointer address for the anyref (which leaks internal browser details, which is very bad for security!)

That's why anyref uses tables: tables don't leak the internal representation (they don't allow you to convert an anyref into a numeric type like a pointer).

The stack doesn't have that problem, because the stack is automatically managed by the wasm engine (wasm programs don't have access to it), so it can't leak internal details.

Would JsValue then hold some sort of (i32 table-identifier, i32 element-index) pair?

Probably not, since an anyref doesn't have to be in a table (it can be on the stack, and I imagine most of the time it will be on the stack).

I guess I kind of assumed it would still end up having to wrap some sort of numeric type so that it can be passed up and down the stack, stored in a Box, cloned, etc.

anyref is an opaque type, which means that wasm knows absolutely nothing about it: it's a black box.

You can cast an anyref into other types, but that only works if the actual runtime value is the desired type. So if an anyref happens to be a JavaScript Number, then you can cast it into an i32 (or other numeric types), but that's not the same thing as converting an anyref into a numeric ID.

So the only way that wasm could convert anyref into a numeric ID is if it always stored anyref in a table (the index in the table would be the numeric ID), or if it always allocated them in a slab (which is the current strategy).

Both of those options imply some performance overhead (it's probably faster to keep anyrefs on the stack as much as possible, so always putting them into a table doesn't sound great for performance).

My understanding is that most of those low-level details will be handled by LLVM (Rust compiles into LLVM, and LLVM then compiles into wasm). I don't know anything about the wasm implementation of LLVM, so I can't answer about what LLVM will do.

I assume that when heap allocating an anyref (such as with Box<JsValue>) that LLVM will put the anyref into a table, but I really don't know.

The other thing I don't quite understand yet is: if JsValueSync is to be something completely separate from JsValue, how would I obtain one?

If you're creating the bindings yourself (using the #[wasm_bindgen] macro), then you should be able to just return JsValueSync (just like how right now you can return JsValue).

For static types, perhaps we could have a #[wasm_bindgen(sync)] attribute, which would tell wasm-bindgen to create a JsValueSync and then wrap it in the proper static type.

That does have some complications, since right now everything is designed under the assumption that it uses JsValue. I'm not sure how to handle that. At a minimum I think JsCast would have to be changed to allow converting JsValueSync into typed values (like how it can convert JsValue into typed values).

For bindings created by other people, we could provide conversions, such as impl From<JsValue> for JsValueSync.

When the conversion is run, it would take the anyref from inside the JsValue and put it into the table, returning a JsValueSync(thread_id, table_index, anyref_index) (or however we decide to represent it).

Since JsValue is !Send + !Sync, that forces the conversion code to run on the thread which created the JsValue, so it is thread-safe.

[Pauan]

See also: #1002

[alexcrichton]

@RSSchermer

(although after studying the WASM proposals I suspect there's a good chance that taking full advantage of these might break the current API anyway?).

Sort of! I think you're right that taking advantage of future wasm feature "in their fullest" is likely a breaking change, but what I was hoping for is that it's a largely cosmetic breaking change rather than "rewrite the world" breaking change. For example switching some types around is pretty easy, but losing a fundamental capability (like "O(1) conversion to an integer from JsValue") runs the risk of making it much harder to migrate in the future.

---

Also FWIW, I can help clarify a few things about the upcoming wasm proposals as well:

• Anyref values indeed cannot be stored in linear memory. Only globals, the execution stack (implicit wasm VM stack machine stuff) and tables.
• Tables cannot be shared between threads.
  • This, coupled with the above, means you can't use wasm to send GC types to other threads, it just can't be done!

---

Also, I can try to help clarify the current state of affairs with anyref and its future with Rust:

• LLVM has no support, nor is it clear how to add support, for anyref. C/C++, AFAIK, have no story at this time to use anyref either. It's actually unclear to me if there's any language lined up to have anyref as a first-class citizen.
• Consequently, we don't know what to do in Rust just yet. It's unclear what the final anyref story will look like. We're working through it!
• An intermediate strategy is implemented in Add experimental support for the anyref type #1002 where it's the exact same wasm-bindgen we have today (API-wise), except that the slab/stack tables are now in wasm instead of in JS, and the imported/exported functions actually talk about anyref. As shown in the PR, though, this requires a lot of intrusive support in wasm-bindgen itself.

@fitzgen and I have discussed a future where anyref is a first-class citizen in Rust-the-language, but it's quite invasive and unclear whether it'll actually pan out. For the time being we're assuming that's not going to happen and #1002 is as good as we're gonna get.

It's clear to me, though, that supporting anyref as a first-class citizen will be an API breaking change to wasm-bindgen. You can think of &JsValue as anyref and JsValue as Box<anyref> (sort of), but the analogy only goes but so far.

[RSSchermer]

Thanks a lot for the detailed replies @Pauan and @alexcrichton, I think I'm finally getting how all the pieces fit together!

I suppose then a lot it is going to depend on how anyref ends up being represented (possibly as something that's on the surface functionally equivalent to a normal heap pointer, or possible as a special NoAlloc kind of type?) and what JsValue then ends up wrapping.

I understand then that it's a bit early to tell what sort of API would end up working, if anything can work at all. I can resort to some transmute hacking for now to rip the idx from a JsValue so that these decisions can be deferred till later.

[alexcrichton]

Ok that seems reasonable to me, I think I'd personally prefer to not provide a public stable API for this, but we can't stop you from transmuting! In the meantime as well I can't imagine how we'd change the representation of JsValue so until anyref happens I suspect such a strategy will continue to work.

@RSSchermer do you think this issue is good enough to close now perhaps?

[RSSchermer]

@alexcrichton Closing would be fine, although given that somehow allocating a JsValue or some derivative of it off the stack seems important, I am still hoping that there'll be a story for this after anyref. However, I'd be happy to open a new issue at that time.