module imports cannot synchronously access exports if it autostarts

[jayphelps]

When using the JS API to dynamically link imports to a wasm module, if that module has an autostart e.g. (start $main) then those imports cannot synchronously access any of the modules exports. Basically you can't have synchronous cyclic dependencies.

There are workarounds:

• Don't autostart
• Have wasm module import everything that is cyclic (depending on what it is, e.g. import the table) instead of it creating and exporting it

const instance = new WebAssembly.Instance(module, {
  env: {
    callIndirect: index => {
      // throws here, instance is not defined yet
      // as the `new WebAssembly.Instance` has not
      // yet returned
      instance.exports.table.get(index)();
    }
  }
});

(module
  (func $callIndirect (import "env" "callIndirect") (param i32))
  (func $main
    i32.const 0
    call $callIndirect
  )
  (func $callback)
  (memory 1)
  (table $table anyfunc (elem $callback))
  (export "table" (table $table))
  (start $main)
)

Demo: example.zip

[jayphelps]

Another possible solution: when invoking an imported function, the spec could say it must be called with the this context of the module instance itself, assuming it has not been bound itself via fn.bind(context)

const instance = new WebAssembly.Instance(module, {
  env: {
    callIndirect: function (index) {
      // `this` is the instance of the module
      this.exports.table.get(index)();
    }
  }
});

[rossberg]

Wasm modules and the JS API aren't really designed to support cyclic linking (a.k.a. recursive modules). A semantics for recursive modules is very tricky to define in the presence of implicit initialisation or imports/exports that aren't just functions. In Wasm you can emulate limited forms of cyclic linking by import indirection through JavaScript functions, but that is more like a workaround. If you can avoid it, I would recommend not introducing cyclic dependencies, since as you note, it does not scale to the full feature set.

[lukewagner]

In Wasm you can emulate limited forms of cyclic linking by import indirection through JavaScript functions

Or, more efficiently, tables :)

[jfbastien]

I agree that it's simpler to disallow cyclic linking for now, but post-MVP we've talked about interfacing with ES6 Modules for real. When we do this, it would be odd to not support cyclicality given that ES6 Modules do (and I mean cycles with and without JS modules).

Does the current design of wasm make it possible to add cyclicality post-MVP?

[rossberg]

Last time I looked, ES6 modules did not simply allow recursive linking either. And ES modules aren't even typed.

There are features in Wasm that make it somewhat difficult already, e.g. globals, tables or memories. It's not clear what you want to allow and what not, especially if you want to avoid the nasty business of constructing global cross-module dependency graphs for all individual definitions. It will be even tougher should we need to allow importing/exporting type definitions (which seems necessary for struct types), because then even validation might generally become recursive.

These are rather inherent problems, I don't think we can take much precaution to prevent them. That doesn't mean it's impossible to add cyclic linking later, but I doubt the result will be pretty.

[lukewagner]

@rossberg-chromium I don't know what you mean by "does not simply allow recursive linking either", but if we're talking about simply allowing two ES6 modules to import each other, I'm pretty sure that is allowed (and implemented).

So I think JS has it hard because, iiuc, you're allowed to run JS in an ES6 module whose top-level script hasn't run and thus you have to very carefully define what that state is. In wasm, I'd be inclined just to have a spec-defined flag on each instance "has my start function been called yet?" and you trap if you attempt to call an export of an instance when the flag isn't set.

Other than that, I assume we'd have all the normal hard problems of cyclic linking that would be solvable, just with effort. @rossberg-chromium anything else?

[jfbastien]

Last time I looked, ES6 modules did not simply allow recursive linking either. And ES modules aren't even typed.

test262 seems to have tests for "cycle" and "circular". Granted some seem to currently fail in V8 due to a spec change.

There are features in Wasm that make it somewhat difficult already, e.g. globals, tables or memories. It's not clear what you want to allow and what not, especially if you want to avoid the nasty business of constructing global cross-module dependency graphs for all individual definitions. It will be even tougher should we need to allow importing/exporting type definitions (which seems necessary for struct types), because then even validation might generally become recursive.

It's not "what I want to allow". It's rather: I'd like wasm to fit nicely with the rest of the web platform. That include ES6 Modules, which we've talked about wasm interfacing with very closely.

These are rather inherent problems, I don't think we can take much precaution to prevent them. That doesn't mean it's impossible to add cyclic linking later, but I doubt the result will be pretty.

Given the conversation, I'm not sure any of us have a clear understanding of how ES6 Modules could / will interface with wasm. This seems suboptimal especially if, by your estimate, the result won't be pretty.

[jayphelps]

Indeed circular imports of function declarations are intentionally supported by ES Modules. There isn't one particular section AFAIK that calls this out, but part of the semantics that make this happen are here: http://www.ecma-international.org/ecma-262/6.0/#sec-resolveexport (and elsewhere). This is one of the many reasons ES modules are superior to CommonJS.

That said, cyclic expressions (including function expressions) are not, for obvious reasons making it impossible under JS semantics.

[lukewagner]

@jfbastien Well, "not pretty", but I don't think @rossberg-chromium is considering the JS definition to be pretty either. I think the more important question would be: would it be any more complicated than what's already in place for ES6 modules.

[jfbastien]

I think the more important question would be: would it be any more complicated than what's already in place for ES6 modules.

Agreed. I don't think we have an answer to this question at the moment.

[lukewagner]

So at least for my part, I investigated this a while back with our ES6 module people and I wasn't able to find any unresolvable contradictions or impedance mismatches (using the "fault if you try to call an export before the start function has been called" semantics). Certainly happy for more people to dig in, though.

[rossberg]

I was under the impression that cyclic linking does not easily work in JS across modules that aren't plain ES source files, e.g. when you need to interop with other JS module formats through custom loader hooks. That would probably affect integration with Wasm as well.

Even for plain ES modules, recursion only works because the language already has various fine-grained dynamic failure modes that the linking mechanism can piggyback on, in particular, temporal dead zones or the undefined value for variable bindings. We don't have (nor want) anything comparable in Wasm, and it would not be applicable to definitions needed at compile/instantiation time anyway (such as types or certain constants).

[lukewagner]

@rossberg-chromium While that is the case for variables which have to execute their initializers in a particular top-level script, iirc, there is a phase before the first top-level script execution that plugs in all the functions across the whole imported graph of modules. I think that is the phase in which static stuff, like types, would be linked.

[rossberg]

@lukewagner, yes, function "hoisting". That works because functions can safely be initialised in any order. That's not the case for e.g. constants. E.g., for Wasm globals you'd need to perform a global topological sort of all individual definitions according to their dependencies. And now add mutual recursion with JavaScript modules to the mix -- I honestly have no idea how that could be handled cleanly.

[lukewagner]

@rossberg-chromium Practically speaking, I think non-function imports (globals/memories/tables) would only be used for load-time-dynamic-linking configurations generated by a toolchain and could all be acyclic by construction. Thus, I think it would be fine to say that these imports are resolved not during function hoisting, but right before the start function is run and that if you try to get a non-function import from a wasm instance whose start function hasn't run, you fail.

[jfbastien]

IIUC this can be fixed when we add support for WebAssembly + dynamic ES modules. I'll close for now since that's on the roadmap.