-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: Higher Order ESM Integration #44
Comments
Why does this use case need to remove all the functionality from this proposal? Why can someone not just use the wasm api directly? |
@devsnek thanks for taking a look. See the "How is this better than workflows today" section where I include a few points here. In particular it is about easy / statically analyzable universal workflows for web assembly integration, which is the primary goal of the ESM integration IMO. |
I get how this is better than no esm proposal at all, I'm just not convinced of how this is better than the current esm proposal. It seems like the foundation of this issue is that some people not having to use fetch is more useful than most people being able to easily consume wasm as a first class module type in js. |
Good point, I should have highlighted this aspect more. The argument here is being able to load all existing Wasm binaries / support all existing Wasm use cases in use with the instantiate APIs today. Eg Wasm binaries with an |
I'd argue that this is more an issue of convention. You can write this JS code today: |
Yes it is important to decide if supporting existing Wasm in use in the wild is a goal or not. Further though, the ESM integration should be designed along with the convention that build tools should output for JS users to support it. Requiring JS users to run Wasm binaries through transformation steps is my concern in creating unnecessary burdens for developers who don't want to think about this stuff. |
Would this allow bundlers and optimizers to remove dead wasm code? What I mean is, imagine that import module from './app.wasm';
const funcs = await WebAssembly.instantiate(module, { env: wasmEnv ));
export { funcs.exportA }; // ignore exportB Would bundlers be able to remove |
assuming well written js you could probably make something that works most of the time. I wouldn't use such a thing in my bundler though. dead code elimination tools for js tend to very quickly give up trying to prove lack of usage when member expressions are involved. |
Yes, I personally recommend the Node.js "exports" field over exports tree-shaking these days https://nodejs.org/dist/latest-v15.x/docs/api/packages.html#packages_package_entry_points. You don't have to optimize out the code you never load. |
I think ultimately JS will want to be able to do both:
Pre-interface-types, I expect toolchains would mostly emit (2) so that they could apply custom JS glue code via |
In #14 we discussed about way to export "this module", I think that's a more elegant than using an evaluator attribute to get the module, like in Just thinking out loud; If Module becomes a first-class type that we can be exported and imported. Couldn't WASI reactors be implemented in an adaptor module that sits between the the ESM integration and the Wasi module. The adaptor module will passthrough the necessary JS values using interface types and exports a |
Revisiting #14, with Module Linking, there's no way for a module to export "itself", only other modules (that were locally-defined or imported), so if I wanted to export a module M, I'd have to wrap it with a module M' which simply nests and exports M. I guess that works, but it seems a little hacky/workaroundy. (I also have a feeling that adding the ability for a module to export "itself" to Module Linking will run into problems, though I can't say exactly what atm.) |
@lukewagner (module
(import "a" (module $a ...))
(module $b ...)
(import "c" (instance $c ...))
(instance $d ...)
(export "e1" (module $a))
(export "e2" (module $b))
(export "e3" (instance $c))
(export "e4" (instance $d))
) so I see no reason why this isn't allowed: (module $a
(export "a" (module $a))
) Therefore #14 has effectively been solved by another proposal entirely. |
An My original suggestion was just that by default it might make sense to treat module-linking modules / modules whose outer module is an A default import treatment like the above would then allow:
Using evaluator attributes it would then still be possible to alter these behaviours to switch either of the above into the alternate mode, but the defaults of the ESM integration become defaults that work for the majority of use cases. My primary concern here is that Wasm binaries output via the conventions of today cannot be executed under the ESM integration in Node.js or browsers without constructing some non-standard semi-private The story I'd just like to see clarified for this ESM integration proposal is what the conventions of integration are for full end to end workflows when this lands without forcing these somewhat cumbersome conventions which users would naturally be incentivised to apply to their workflows otherwise. If environments like Node.js and Deno (or browsers) want to apply the ESM integration in the next couple of years, these types of workflow conventions start to become actual engrained patterns otherwise, unless we should be sure to change the recommended import conventions before then in order to ensure better alignment. |
@00ff0000red Nice job digging in! That would almost work, but the validation rules only allow referring to preceding modules, with the specific intention of preventing cycles like that. (Particularly with type imports/exports, cycles introduce serious complications.) @guybedford One side note is that "module linking modules" shouldn't be a distinct "kind" of module; module linking just gives you new ways of defining and instantiating (existing) core wasm modules. But I think your point stands w.r.t adapter modules, which are a different kind of module. Having separate defaults (for core vs. adapter modules), with later evaluator attributes to flip the behavior, seems like a practical potential solution to the "env" problem you're talking about. |
I feel like this is an irrelevant argument for adoption of Wasm into ESM. Just like with the initial adaption of ESM into ES, people didn't just convert their code to a module, it usually had to be rewritten or more heavily changed to make use of ESM. Some workflows still haven't been adjusted to use ESM, some still preferring to downlevel their code to static linkage anyway. Similiarly, when ES6 introduced "use strict," it wasn't just slapped on old code, it would simply break old code. To assume that one could just import a non-ESM Wasm modules seems wrong. If the core Wasm were exported as a |
When I first wrote this proposal I wasn't clear at the time that the distinction here would be based on the adapter module instead, thanks for explaining.
I guess the concern then is if having a semantic difference between core and adapter modules will introduce more complexity / cognitive overhead to these workflows or less. Perhaps the deciding question here in terms of what the practical workflows will be is really how much we can lean on evaluator attributes as being an available solution within the next two years / similar timeframe as shipping of the ESM integration? @littledan wondering if you have any thoughts on that. In terms of the natural conventions, I previously mentioned import maps and a nested private |
I like the idea of keeping the "default" Wasm/ESM semantics as is in this proposal, and having some kind of additional syntax to opt-in to getting an uninstantiated module. About when evaluator attributes will be available: this all depends on when people bring clear use cases to TC39 and champion the proposal. Import assertions took less than a year to get from nothing to Stage 3, so I don't think that evaluator attributes will necessarily take very long, if people put in the work. I don't plan to champion the evaluator attributes proposal personally, but I am happy to mentor others to work on it. This proposal is really blocking on implementation work, whereas evaluator attributes need this design work, so they may take longer. Is there some reason that they should be released in a similar timeframe? |
Thanks all for the engagement here, it's been very helpful. To summarize my opinions on this topic:
This does give me a new sense of the importance of evaluator attributes and I would be glad to get involved in assisting evaluator attributes however I can, although I likely don't have bandwidth for the next couple of months myself. |
The last few issues on this repo have brought up the security question of whether or not Web Assembly provides a more secure execution environment in comparison to JS executions.
In addition, there've been much discussion over the exact workflows for the Web Assembly start fuction and binding process in terms of ensuring that the ESM import workflows provide the major use cases achieved with the declarative
WebAssembly.instantiate
APIs that are currently used today in Web Assembly applications.Originally in this repo, @alexcrichton suggested in #14 an API for the Web Assembly ESM integration to support importing a
WebAssembly.Module
object directly, in order to allow more easily and flexibly working with a compiled module.Along the lines above, I'd like to propose changing the ESM semantics of importing Web Assembly in order to better achieve security and use case flexibility for Web Assembly applications.
Proposal
Web Assembly offers some highly compelling execution security properties in providing a strictly defined secure execution sandbox, down to the imported bindings provided to it.
By default the ESM integration would not naturally benefit from these security properties since it permits arbitrary JS imports from Web Assembly modules.
Instead of following the naive ESM integration, the proposal would be for all Web Assembly modules that are imported, to be imported as compiled
Module
objects, leaving the binding process to the JS wrapper code entirely:The imported value of the Web Assembly module is only a
{ default: Module }
ES module, that can then be instantiated with a JS call toWebAssembly.instantiate
:By only providing the uninstantiated
Module
, this supports a number of useful properties:How is this better than workflows today
The first criticism of this proposal might be - what benefit exactly does the ESM integration provide at all, if this is the case?
How is this a benefit over just fetching and compiling the Wasm module directly? Eg via:
And this is true, the benefit is exactly in just reifying the above pattern into an import pattern, including:
fetch
global. This means that Web Assembly instantiations are inconsistent and there exists no "universal" easy pattern.import.meta.url
relative fetch pattern is still very new and not many tools support it. Very few JS build tools today will properly detect thatapp.wasm
is a binary that needs to be relocated included in the build folder in the above pattern.Interaction with Import Assertions
Since importing Wasm becomes a non-executing and secure operation, it then makes sense that a Wasm module import assertion can verify this property:
This fits the definition of import assertions in being entirely validation based, and importantly not splitting the interpretation of the module depending on the assertion / mode used.
Interaction with Module Linking Proposal
The module linking proposal provides a way for Web Assembly to handle instantiation and binding setup between multiple Web Assembly modules to provide a richer end-user API directly.
These Web Assembly modules can effectively be thought of as a new type of higher-order Web Assembly module for these wiring needs.
In the ESM integration for the module linking proposal, it would be possible to define that these modules are treated differently, such that you would get the exports and imports applying as one might expect with a JS import or the naive ESM integration.
These modules would lose the sandboxing properties so the import assertion would not apply to these forms of modules making the distinction between these two different ESM integration cases clearer.
The text was updated successfully, but these errors were encountered: