How does this work with Wasm/ESM integration?

[littledan]

https://github.com/webassembly/esm-integration

One mechanism I could imagine: maybe Wasm modules can access these string built-ins (and all future built-ins) as default-present built-in modules. Note that these were rejected for JS, though, and I'm not sure what the rationale for different decisions. (Or, are we leaning against Wasm/ESM integration as previously conceived?)

[eqrion]

Some related discussion in tc39/proposal-source-phase-imports#56. I think that explains how we could express compile-time imports in a future with ESM-integration.

Are you referring to https://github.com/tc39/proposal-built-in-modules/?

I don't think we strictly need a default-present built-in module to make this work, you could write a simple JS module the re-exports the values from the WebAssembly.String and compile-time import that module from wasm.

However, default-present built-in modules would make it easier and could be a good use-case for it, if it was ever reconsidered.

[guybedford]

I think what @littledan is referring to here is the default ESM integration behaviour for import { exportedFunction } from './mod.wasm'.

Certainly import maps offer a way out here, but that does naturally lead to agreeing on the names of these imports when you have more than one Wasm module in an HTML page, and then that leads to wanting to define a scheme etc etc.

Reopening the builtins discussion could be tricky though certainly, so understood if this is a non goal. But it's worth noting that a separate section might be a way to define builtin definitions that must be provided without hitting the ESM integration / builtin namespace concerns.

I'll respond on the source phase interactions in the other thread.

[eqrion]

I realized I posted a response to this on tc39/proposal-source-phase-imports#56 (comment) when I really should have posted this here so that parties following this repo could see it. Reposting it below:

Okay, coming back to this after discussing this with several folks offline.

I didn't fully understand the issue because I thought the issue was with how to specify certain wasm imports as being 'compile-time' when using ESM-integration. But the bigger issue is that even if you have that, you need that compile-time import to get some value from another module's exports and that doesn't fit in the ESM module phases (evaluation comes last) without extending it quite a bit.

Proposed new direction

So here's another option (restated and tweaked from @guybedford and others) that supports the use-case of js-builtins in wasm, without using compile-time imports:

1. Add an option to the WebAssembly JS-API compilation endpoints (module constructor, compile, compileStreaming, etc):

dictionary WebAssemblyCompileOptions {
    optional sequence<USVString> builtinModules;
}

Every string in builtinModules must be a host-recognized 'builtin-module', and the list must not have duplicates. Every 'builtin-module' that's provided is available as an import when compiling the WebAssembly module.

Any module import that refers to something from a builtin-module is checked eagerly when compiling and then is no longer needed to be provided when instantiation happens (same as with compile-time imports). They're 'eagerly' applied and this allows us to specialize codegen to them.

2. Define a 'wasm/string' builtin-module, basically the same as the proposal currently has it in the WebAssembly namespace

3. In a future with ESM-integration, builtin-modules are available to be imported and because they're guaranteed to always resolve to the host we can recognize and specialize codegen to them while we compile the module for instantiation later.

Advantages

1. No longer need any restrictions around serializability/shareability that compile-time imports introduced. Anything that's within a builtinModule is reasonably expected to be serializable and shareable.
2. Coherent story for working with ESM-integration in the future

Disadvantages

1. Unclear if we could make Web features available in the wasm-builtin-modules or only just JS features that are purely computational and don't represent a 'capability'. Even if true, this may not matter as we may only care about computational features?

Open question

Would source importing a wasm module that imports a wasm builtin-module have the same 'eager' behavior as in the JS-API above?

If the answer is yes, then this would seem to make certain import namespaces behave differently, which is not ideal. If the answer is no, then we can't easily specialize codegen to these builtin modules, which is the whole point of this proposal.

Maybe it could be optional using an import attribute of some kind to opt-in to eagerly applying some imports?

[eqrion]

Now that I've paged more of this back after travel, I believe I missed a subtlety that answers my open question.

The tweak to above would be to not think of these as compile-time imports, but as normal imports still provided at instantiation, only they are from a reserved namespace that users cannot override. Those combined allow for engines to know at compile time what the eventual import will be at instantiation time.

The purpose of the new flags in JS-API is just to opt-in to reserving the 'wasm/' module namespace (for backwards compat), not as a hardcoded compile-time import list.

So with the JS-API when the user opts-in, we wouldn't apply eager type checking as the imports aren't actually provided until instantiation. At instantiation we disallow the user imports object from specifying the 'wasm' module namespace as that has been reserved to be provided by the host.

With esm-integration, source-phase imports would also not have any eager type checking or 'fusing', but when instantiated would have the same behavior of disallowing the user from providing the 'wasm' namespace and only allowing the host to provide it.

[conrad-watt]

The tweak to above would be to not think of these as compile-time imports, but as normal imports still provided at instantiation, only they are from a reserved namespace that users cannot override. Those combined allow for engines to know at compile time what the eventual import will be at instantiation time.

This seems somewhat more messy to me - in previous debates on string builtins we kind of believed it would always be an option for users not to provide the builtin and instead somewhat muddle through with their own polyfill. Also, if the engine is able to generate the right code at compile time because it "knows" what import will be provided at instantiation time based on the reserved name, what's the point of actually providing the import at instantiation time? Is the main motivation for this approach to better fit ESM's model of how Wasm instantiation works?

Another concern - this means that we have to be more careful about our builtin names - if some already-deployed user code shadows a builtin name we want to use, we'd have our own version of smooshgate.

EDIT: actually, I guess if the flag reserves the whole wasm/ module namespace as described above, this last concern is mitigated, although there's less granularity to polyfill some builtins but not others

[conrad-watt]

To unpack my thoughts on this a little more - this feels like the proposed JS-API flag semantically means "make XYZ compile-time imports ambiently available at compile-time, and assume they will be used to satisfy the corresponding imports in the module under compilation". Then, at instantiation-time, we require redundant shuffling of "import tokens" that don't actually have semantic meaning, mirroring the decisions already made at the point the JS-API flag was set. It doesn't seem a stretch to instead say that setting the JS-API flag removes the satisfied imports from the compiled module, and then we're back to straightforward compile-time imports - the flag just means "bring this pre-defined set of pre-imports into scope".

I'm not very familiar with ESM-integration so apologies if I'm asking the wrong questions, but is there something that's stopping this same (latter) story from working? Do we have to expose builtins in the form of a module for import at instantiation-time or is there somewhere that an analogous flag could be added?

[eqrion]

@conrad-watt The mental models of both approaches are very similar. The biggest difference is around whether there is early type checking and whether the imports are shown in reflection (Module.imports()). In both, you would not need to provide the import values again during instantiation (you could think of them as ambiently available in the 'reserved namespace' model).

I believe that either could work in ESM, I just mentioned the reserved namespace approach as it makes these imports less special. You don't have to do early type checking, or remove the imports, etc.

However, there could also be an advantage to treating these builtin imports as special. It could give us more of a rationalization for making them even more special if we wanted to bind them to specific wasm memories for certain builtins.

[eqrion]

Compile-time imports were replaced with a builtin modules approach in #8.

So with plain esm-integration, importing from 'wasm' 'js-string' would resolve the host provided builtin module by default. If you wanted to polyfill the module, you could use import maps to override this.