Largely rewrite the explainer

[littledan]

The new explainer is opinionated, making the case for Variant B with
no deadlock prevention.

[littledan]

Note, the examples here have sort of regressed in realistic-ness compared to Myles' previous explainer. Help in writing better examples (and any other improvements to this text) would be greatly appreciated.

[ljharb]

This seems very spooky-action-at-a-distance to me. I can add an await in my module, and alter the execution order of all the modules importing me?

[littledan]

This sounds like an argument against the feature as a whole, rather than about Variant A vs Variant B. Did TC39 fail to consider this properly when proposing for Stage 2? (I encourage you to file an issue about your concerns; they are more likely to be lost here.)

[ljharb]

Yes, this problem was brought to my attention right after the stage 2 meeting, and since it's never been brought back to the committee, and since #42 never happened, i've not had a chance to air my concern fully. Had this occurred to me prior to that meeting, I'd have objected to stage 2.

[littledan]

Is it documented clearly in any particular issue?

[ljharb]

Nope; I think it's the sort of thing that needs talking through before I'm able to verbalize it in an issue well.

[domenic]

I would strongly suggest raising an issue with a concrete example, as I don't understand the concern as raised. I think an issue with a textual example will be much clearer than trying to do so verbally.

[ljharb]

I very much disagree this is popularized; I have to tell people to use Promise.all to avoid unnecessarily serializing their code with await multiple times a day, every day, since async/await became popular. It's a commonly fired footgun, and education has not yet been sufficient to avoid it.

[littledan]

Interesting; maybe file an issue about it? Also seems like an argument about the feature as a whole.

[ljharb]

I'm glad to read this; this is an important guarantee to be preserving.

[littledan]

Yeah, I think this property was never in doubt, and there were just some communication issues about it.

[domenic]

You could add something like "This is similar to [an extension of?] how in the current system none of the statements in usage.mjs will execute until awaiting.mjs is loaded and all of its statements has executed."

[domenic]

I might approach this from a different direction (or, start out with a different explanation). Something like:

Currently, a module waits for all of its dependencies to execute all of their statements before the import is considered finished, and the module's code can run. While introducing await, we want to keep this property: dependencies should still execute through to the end, even if you need to wait for that execution to finish asynchronously. The natural way to think of this is as if each module exported a promise...

[domenic]

I would strongly suggest raising an issue with a concrete example, as I don't understand the concern as raised. I think an issue with a textual example will be much clearer than trying to do so verbally.

[domenic]

Is it worth including an export const dPromise = at the beginning of this line, to show how it propagates?

[domenic]

I think "dependency" is much clearer than "exporting module".

[domenic]

Suggested change

	If one module wants to declare itself dependent on another module, for the purposes of waiting for that other module to complete its top-level `await` statements before the module body executes, it can delcare that other module as an import.
	If one module wants to declare itself dependent on another module, for the purposes of waiting for that other module to complete its top-level `await` statements before the module body executes, it can declare that other module as an import.

[domenic]

Use separate code blocks for each file to make it a bit clearer.

[domenic]

I would extend this example to show how if x.mjs logs before it awaits, that log ("X1") comes before "Y". So, execution order is preserved up until an await.

[littledan]

Thanks for your constructive review, @domenic . These all seem like good ideas, so I've integrated changes from all of your suggestions.

[bergus]

default is a keyword and cannot be used as an identifier name here

[littledan]

Thanks, fixed.

[bergus]

This still sounds a bit like Variant A, doesn't it? Considering

// u.mjs
console.log("u1");
await new Promise(r => setTimeout(r, 1000));
console.log("u2");

// x.mjs
import "u";
console.log("x");

// y.mjs
console.log("y");

// z.mjs
import "x";
import "y";
console.log("z");

Without TLA in u.mjs, we'd get u - x - y - z.
But with the TLA in u.mjs, I would expect to get u1 - y - u2 - x - z. Does y really need to wait for x, because of the established traversal?

[littledan]

y doesn't need to wait for x. I agree with your expectations of ordering, and this text tries to explain them, alongside @guybedford's specification text for them in #33. Could you point a little more closely to what part of this text made you think otherwise?

[bergus]

Thanks for the confirmation that my expectations aren't off.
I was confused by "modules always start running in the same post-order traversal established in ES2015", which seems to suggest that y would need to start running after x. The "After a top-level await is reached, control may be passed to start the next module in this traversal order" seemed to contradict that a bit, but I thought it would be geared towards the x-y-z example in the previous section ("block the import of an adjacent module"), and there x still starts before y.

[bergus]

Maybe a few tiny changes could clear this up:

Regardless of whether top-level await is used, modules initially start running in the same post-order traversal established in ES2015: execution of module bodies starts with the deepest imports, in the order that the import statements for them are reached. After a top-level await is reached, control will be passed to start the next (first?) ready-to-run module in this traversal order.

[littledan]

Thanks for clarifying. What do you think of the version I just uploaded?

[bergus]

I like it :-)

[bergus]

Another problem: you need to separate declaration from initialisation. One could not write export const example = await …, but rather has to declare a top-level export let/*sic!*/ example and then initialise it in the IIAFE.

So while this would indeed be a proper workaround, nobody would use it. People, who already can often not distinguish destructurable objects from aliased import/export bindings, would instead default-export a promise that fulfills with a "module" object, and use

import x from "x";
export default (async () => {
    const {xa, xb} = await x; // the parts of the import
    return { // the module "exports"
       exportA() { … }
       exportB: …
    };
})();

[littledan]

Yeah, I agree that that is another unfortunate pattern that could emerge. Both patterns are bad; we could document both, but I thought this one would make the explainer clearer as it's closer to the solution.

Do we have any evidence that either of these two patterns is catching on? The GitHub search evidence that was previously used in the explainer seems rather weak.

[littledan]

You can see the switch from let to const and the combination of declaration and definition in the code sample I included. Would it really help to spell this out with further text?

[bergus]

I have no idea how to use Github search effectively for design patterns. This approach is however what I myself have recommended before, as it cleaner (no race conditions, less chance of forgetting the promise, simpler syntax) than the alternative.

Maybe introduce the closer-to-the-solution pattern only in the "Semantics as desugaring" section below?

[littledan]

What do you think of the text I just uploaded, which includes your pattern?

[bergus]

Thanks, perfect!

[zenparsing]

Thanks - this update to the explainer is very helpful.

[zenparsing]

I'm curious about the execution order for the example below, given the "always Promise.all" behavior described further down:

// a-1.js
console.log('a-1');

// a-2.js
console.log('a-2');

// a.js
import './a-1.js';
import './a-2.js';
console.log('a');

// b.js
console.log('b');

// main.js
import './a.js';
import './b.js';
console.log('main');

The current ES semantics will result in

a-1
a-2
a
b
main

With the suggested Promise.all semantics, what will be the result?

[bergus]

It should - no, must - stay the same, as there is no await anywhere in the example and we want to be backwards-compatible.
I think with Promise.all semantics, the modules will load in the same order as previously, will wait for their dependencies (which are synchronous, so immediately fulfilled promises), and then execute in that same order as promise tasks run in the same order they were queued in. The only difference is that they will be run in microtasks, see #43, but that doesn't affect order.
Oh, wait, you are concerned that a needs to wait for more promise resolution stuff than b, and might be evaluated after it?

[littledan]

Right, the semantics of this proposal are that the order would stay the same as what you describe. The Promise.all affects how JS waits for dependencies, not how it launches their execution. (However, there is an observable change in that some turn(s) through the job queue would be taken in between modules.)

Seems like this property has been confusing and difficult to explain. Your help in pinpointing what causes this confusion would be appreciated.

[zenparsing]

@bergus That's right. It seems from the explainer that the result will now be:

a-1
a-2
b
a
main

Because "a" has to defer its body evaluation until Promise all'ing "a-1" and "a-2", which will push its evaluation job out past "b".

Does that make sense?

[bergus]

@littledan not sure whether it's just a mistake in my naive attempt at translating zenparsing's example to the "Promise.all semantics" and the actual semantics in #33 handle this correctly (I haven't dived into the details yet), or whether it is an actual issue, but

const a1 = Promise.resolve()    .then(() => console.log('a-1'));
const a2 = Promise.resolve()    .then(() => console.log('a-2'));
const a  = Promise.all([a1, a2]).then(() => console.log('a'));
const b  = Promise.resolve()    .then(() => console.log('b'));
           Promise.all([a, b])  .then(() => console.log('main'));

does output the logs in the wrong order due to Promise.all([a1, a2]) queuing more promise jobs than the Promise.resolve that b waits for.

(edit) To clarify: this behaviour is not what we want. @zenparsing thanks for the confirmation.

[littledan]

@zenparsing Oh, do you mean it would be pushed out because of several turns through the job queue would make "a" land later than "b" (in a sort of "race") where the delay isn't really triggered by waiting external resources?

Maybe what we're stumbling towards is that we should drain the microtask queue between sibling execution!

[bergus]

Maybe, but explicit queue draining would be a wart imo.
I guess it's easier to have a "synchronous" version of Promise.all, that checks whether all of its arguments are fulfilled already and if so immediately returns a result, falling back to the standard asynchronous procedure otherwise. (Yes, I know about Zalgo, but it could be warranted here). Even better if made explicit by using a result type of SynchronouslyEvaluated<ModuleRecord> | TlaEvaluated<Promise<ModuleRecord>>.

The alternative would be to make the traversal explicit and maintain a graph or queue, instead of doing it implicitly in the recursive algorithm. This might even give way to explicitly allow for concurrent (pre)loading of module code.

[littledan]

Let's continue this discussion in #47 . Variant B has been proposed for a long time, and isn't introduced by this PR.

[MylesBorins]

LGTM. I'll land in 24 hours unless there is an explicit objection. We can continue to dig in and augment as improvements are identified

[MylesBorins]

Thanks @littledan !