-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Communication via TypedArrays and ArrayBuffers #89
Comments
This is probably pushing the limits of what can be expected from a WebView, but it would be totally wild if you could use SharedArrayBuffers to share memory between JavaScript and C++. Then, you could start pushing the WebView into unusual domains like VR, video processing, high-end games, etc. Though, initially, just a bulk binary data copy would be enough for me. |
This is really interesting. The general need of sharing binary data between native and JS makes sense. I think there are two quite different approaches you mentioned, we can discuss different scenarios and figure out what makes sense.
Stepping back a little to your printing example, why are you trying to pass the data to native to call Windows printing API (and which API are we talking about here)? Can you use window.print web API? |
My app in a vector drawing tool, so I need complete control of the printing process to get good quality printing where things are properly scaled to the page size and match the printer resolution and whatnot. The window.print web API just doesn't offer deep enough native integration to work right. My understanding is that all the Windows printing APIs have been deprecated except for the historical GDI Print API and the newest UWP printing stuff. I'm targeting the GDI printing stuff. Regardless, I'm using a vector graphics model that it too advanced to be represented with older vector graphics technologies like pdf or SVG or WMF or whatever, so I need to rasterize it myself. The printing is just an example though. I've also been thinking about exporting animations as video from JavaScript, or doing CUDA processing on data, and other stuff, but I've been too afraid of that data transfer overhead. |
Makes sense, thanks for sharing the scenario. I think we'll look into it (but maybe not in the immediate month or so due to priorities like shipping a .NET preview). |
You could try intercepting the data via |
A new API has been developed for this scenario. Please check it out (the API is supported in current stable WebView2 Runtime, version 107). |
The documentation should be more clear about whether you receive an ArrayBuffer or SharedArrayBuffer on the JavaScript side. The current terminology of “shared buffer” can cause confusion. Is it an ArrayBuffer that is shared with native but JS can only transfer ownership to web worker threads? Or is it a SharedArrayBuffer that is shared with native and can also be shared with web worker threads? Ideally, it should outright say, “this is not a SharedArrayBuffer object” or something. |
It might also be useful to show an example of passing data from JS to native. I assume it’s not possible to create a shared buffer from JS, so the JS side would first have to message native side to create such a buffer and pass it into JS where it can be filled with data? |
It is ArrayBuffer, not SharedArrayBuffer. The ArrayBuffer is shared between native and JS side. On JS side, you have to transfer ownership to worker thread if we want to access it from worker thread. |
Is there any way to trigger atomic locks (or any other mechanism of blocking concurrent access)? JS has Atomics, but I haven't seen anything in those docs that would tell me how I can coordinate with the UI thread from the host and avoid race conditions. |
The safest way would be to access the buffer and post a message to the other side to notify that the data is produced/consumed. If we really want to access the buffer at about same time, our limited testing suggests that Atomics can be used from JS side while std::atomic can be used on native side. Test it in your scenario to verify. |
Hey @LiangTheDev , I could not find a way to use it with UWP in the latest stable release. Could you please help me with it? |
The API is still in experimental, so you would have to use a pre-release SDK. As WinUI3 seems to only support public WebView2 SDK, you might have to wait for the API to go out of experimental, or switch to WinUI2 which allows using pre-release WebView2 SDK. |
Is it possible to implement buffer pool using this shared buffer mechanism? |
What is the current status of this feature? Is it C++ only, or is it available for .Net as well? It would be sooo cool, if ArrayBuffer type would be automatically marshaled as byte[] both ways, like strings or numbers are (in js-to-host and host-to-js calls or in PostMessage). |
Look at the SharedBuffer class: |
@mikeduglas , excellent, thanks! This takes care of sharing from the host to JS. Is there anything similar to go from JS to host? (of course, one can invent a convoluted example where JS would request a buffer from the host, the host would supply it, then JS would fill it with data, and so on, but that sounds very ineffective, complicated, and error-prone). |
The "convoluted example" is currently the way to initiate data sharing from JS to host. There is no other APIs to do it easier. Note that the buffer can be reused multiple times and hopefully that would reduce number of this round trip communication. |
Would you be able to expand on this @LiangTheDev? Given that the WebView2 shared buffer is exposed as an JS ArrayBuffer type rather than a JS SharedArrayBuffer type, would it not exclude using JS Atomics? |
According to https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Atomics:
|
Unfortunately then it seems only Firefox implements Atomics on Currently I'm trying to avoid using |
Try it out. It actually works for WebView2's posted ArrayBuffer in our test. |
I've created a sample project here: https://github.com/aaronbloom/WebView2/blob/master/WebView2Atomics/MainWindow.xaml.cs#L49 I encounter a TypeError when using the WebView2 shared buffer with the JS Atomics API. Hopefully I've just misused the API. |
Got what you are saying. There are certain Atomics APIs like wait that are only work with views to SharedArrayBuffer. Those APIs would not work. The other APIs like compareExchange should work. If the scenario requires wait, then it would indeed not work. |
Thanks for the update. I've added each Atomic method into my sample, and it matches with what you are saying. Output:
Given that |
Modern high-performance JavaScript makes a lot of use of typed arrays and array buffers. It's simply too slow to transfer data around using JSON and strings. It would be nice if there was a way to do bulk data transfers between c++ <-> JavaScript.
For example, I currently have an Electron app that does some printing in Windows. Printing one page involves generating 8.5in x 600dpi x 11in x 600dpi x 3 colors = 100MB of data that needs to be transferred from JavaScript to C++ so that it can be fed into Windows' printing APIs. I can generate that amount of data relatively quickly in JavaScript, but without a dedicated API for transferring binary data from JavaScript to C++, the data transfer can end up being a real bottleneck (I need to encode the data as a string using base64 and then send it over, and that's slow and unresponsive, so I need to break it up into separate units of work, etc.).
AB#28510494
The text was updated successfully, but these errors were encountered: