Tracking issue: require(esm) #52697

joyeecheung · 2024-04-25T21:25:06Z

Before it's unflagged

Figure out default export interop with transpilers, either adding __esModule to required ESM on our end (module: add __esModule to require()'d ESM #52166), or transpilers update themselves to check the result returned by require():
conditional exports for module, regardless of whether it's loaded by require or import. Something like module which is recognized by Webpack and Rollup would be good (maybe this doesn't need to block unflagging, but should be done before stablization) module: implement the "module-sync" exports condition #54648
Move experimental warning to where require() is actually handling a ESM

Before it is promoted to be stable:

Implement detection module: support ESM detection in the CJS loader #52047
Some marker to customize the default exports returned by require(esm) module: support 'module.exports' interop export name in require(esm) #54563
Support module customization hooks doc: add synchronous hooks proposal loaders#198

Nice-to-haves:

Bug fixes & changes:

Related features that interoperate with require(esm) and need to be considered when being backported together:

For v22.x backport (see a summary of regression analysis in #55217 (comment))

The text was updated successfully, but these errors were encountered:

GeoffreyBooth · 2024-04-30T19:16:14Z

@nodejs/loaders

jakebailey · 2024-05-07T22:00:30Z

I just wanted to note it here, but it would be super super awesome if (once stable) this were backported to Node 20/22 or even Node 18 if still in support. I'd love to be able to propose a change to switch TypeScript to ESM (given I have it working without breaking CJS consumers), but the time horizon of Node 22 being the oldest supported version is pretty daunting.

It also seems like there is a hacky way using multiple entrypoints that could allow for TS to grab Node's builtins conditionally without #52599/#52762, though none of that is possible without require(ESM), of course.

Even without TypeScript's use case, I think the feature itself is a really important one for the ecosystem. Backporting would really make ESM changeovers a lot less painful.

Andarist · 2024-05-07T22:07:50Z

IIRC from some Twitter threads - there is a plan to backport this once the feature stabilizes.

joyeecheung · 2024-07-11T23:16:42Z

Regarding the conditional exports, @guybedford suggested to implement just the module condition that Rollup and Webpack already recognize. I have a WIP but need to do some testing which involves many combinations of test cases 😵‍💫 but will also do some npm crawling to see if it can be picked up/is in conflict with any existing popular packages.

GeoffreyBooth · 2024-07-11T23:41:11Z

@guybedford suggested to implement just the module condition that Rollup and Webpack already recognize.

The module top level field, or within exports?

Personally I think require-module makes more sense, as it would complement the existing require and import keys that Node supports.

joyeecheung · 2024-08-29T22:54:36Z

Opened PR for "module" in #54648

Personally I think require-module makes more sense, as it would complement the existing require and import keys that Node supports.

If we are starting from scratch, yes, but then the "module" condition has already been adopted by bundlers that support require(esm) in the wild, so it seems better to go along with the existing convention. See https://gist.github.com/sokra/e032a0f17c1721c71cfced6f14516c62

This patch implements a "module-sync" exports condition for packages to supply a sycnrhonous ES module to the Node.js module loader, no matter it's being required or imported. This is similar to the "module" condition that bundlers have been using to support `require(esm)` in Node.js, and allows dual-package authors to opt into ESM-first only newer versions of Node.js that supports require(esm) while avoiding the dual-package hazard. ```json { "type": "module", "exports": { "node": { // On new version of Node.js, both require() and import get // the ESM version "module-sync": "./index.js", // On older version of Node.js, where "module" and // require(esm) are not supported, use the transpiled CJS version // to avoid dual-package hazard. Library authors can decide // to drop support for older versions of Node.js when they think // it's time. "default": "./dist/index.cjs" }, // On any other environment, use the ESM version. "default": "./index.js" } } ``` We end up implementing a condition with a different name instead of reusing "module", because existing code in the ecosystem using the "module" condition sometimes also expect the module resolution for these ESM files to work in CJS style, which is supported by bundlers, but the native Node.js loader has intentionally made ESM resolution different from CJS resolution (e.g. forbidding `import './noext'` or `import './directory'`), so it would be semver-major to implement a `"module"` condition without implementing the forbidden ESM resolution rules. For now, this just implments a new condition as semver-minor so it can be backported to older LTS. Refs: https://webpack.js.org/guides/package-exports/#target-environment-independent-packages PR-URL: #54648 Fixes: #52173 Refs: https://github.com/joyeecheung/test-module-condition Refs: #52697 Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Jan Krems <jan.krems@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com>

This unflags --experimental-require-module so require(esm) can be used without the flag. For now, when require() actually encounters an ESM, it will still emit an experimental warning. To opt out of the feature, --no-experimental-require-module can be used. There are some tests specifically testing ERR_REQUIRE_ESM. Some of them are repurposed to test --no-experimental-require-module. Some of them are modified to just expect loading require(esm) to work, when it's appropriate. PR-URL: #55085 Refs: #52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: LiviaMedeiros <livia@cirno.name> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Filip Skokan <panva.ip@gmail.com> Reviewed-By: Michael Dawson <midawson@redhat.com> Reviewed-By: Richard Lau <rlau@redhat.com>

This patch implements a "module-sync" exports condition for packages to supply a sycnrhonous ES module to the Node.js module loader, no matter it's being required or imported. This is similar to the "module" condition that bundlers have been using to support `require(esm)` in Node.js, and allows dual-package authors to opt into ESM-first only newer versions of Node.js that supports require(esm) while avoiding the dual-package hazard. ```json { "type": "module", "exports": { "node": { // On new version of Node.js, both require() and import get // the ESM version "module-sync": "./index.js", // On older version of Node.js, where "module" and // require(esm) are not supported, use the transpiled CJS version // to avoid dual-package hazard. Library authors can decide // to drop support for older versions of Node.js when they think // it's time. "default": "./dist/index.cjs" }, // On any other environment, use the ESM version. "default": "./index.js" } } ``` We end up implementing a condition with a different name instead of reusing "module", because existing code in the ecosystem using the "module" condition sometimes also expect the module resolution for these ESM files to work in CJS style, which is supported by bundlers, but the native Node.js loader has intentionally made ESM resolution different from CJS resolution (e.g. forbidding `import './noext'` or `import './directory'`), so it would be semver-major to implement a `"module"` condition without implementing the forbidden ESM resolution rules. For now, this just implments a new condition as semver-minor so it can be backported to older LTS. Refs: https://webpack.js.org/guides/package-exports/#target-environment-independent-packages PR-URL: nodejs#54648 Fixes: nodejs#52173 Refs: https://github.com/joyeecheung/test-module-condition Refs: nodejs#52697 Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Jan Krems <jan.krems@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com>

This unflags --experimental-require-module so require(esm) can be used without the flag. For now, when require() actually encounters an ESM, it will still emit an experimental warning. To opt out of the feature, --no-experimental-require-module can be used. There are some tests specifically testing ERR_REQUIRE_ESM. Some of them are repurposed to test --no-experimental-require-module. Some of them are modified to just expect loading require(esm) to work, when it's appropriate. PR-URL: nodejs#55085 Refs: nodejs#52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: LiviaMedeiros <livia@cirno.name> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Filip Skokan <panva.ip@gmail.com> Reviewed-By: Michael Dawson <midawson@redhat.com> Reviewed-By: Richard Lau <rlau@redhat.com>

voxpelli · 2024-10-03T12:53:10Z

Raised a question on Twitter to @joyeecheung on my conclusions in https://github.com/voxpelli/investigation-esm-require where it seems like Node 22.9.0 may unintentionally allow some ESM-files to be loaded even without the flag: https://twitter.com/voxpelli/status/1841818608713826693

Mentioning here for sake of completeness, if deemed a correct observation a proper issue will be created

voxpelli · 2024-10-04T07:46:34Z

The above resulted in a PR to fix it: #55250

This patch implements a "module-sync" exports condition for packages to supply a sycnrhonous ES module to the Node.js module loader, no matter it's being required or imported. This is similar to the "module" condition that bundlers have been using to support `require(esm)` in Node.js, and allows dual-package authors to opt into ESM-first only newer versions of Node.js that supports require(esm) while avoiding the dual-package hazard. ```json { "type": "module", "exports": { "node": { // On new version of Node.js, both require() and import get // the ESM version "module-sync": "./index.js", // On older version of Node.js, where "module" and // require(esm) are not supported, use the transpiled CJS version // to avoid dual-package hazard. Library authors can decide // to drop support for older versions of Node.js when they think // it's time. "default": "./dist/index.cjs" }, // On any other environment, use the ESM version. "default": "./index.js" } } ``` We end up implementing a condition with a different name instead of reusing "module", because existing code in the ecosystem using the "module" condition sometimes also expect the module resolution for these ESM files to work in CJS style, which is supported by bundlers, but the native Node.js loader has intentionally made ESM resolution different from CJS resolution (e.g. forbidding `import './noext'` or `import './directory'`), so it would be semver-major to implement a `"module"` condition without implementing the forbidden ESM resolution rules. For now, this just implments a new condition as semver-minor so it can be backported to older LTS. Refs: https://webpack.js.org/guides/package-exports/#target-environment-independent-packages PR-URL: #54648 Fixes: #52173 Refs: https://github.com/joyeecheung/test-module-condition Refs: #52697 Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Jan Krems <jan.krems@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com>

targos · 2024-10-05T09:38:54Z

It looks like npm itself triggers the warning:

$ make lint
cd tools/eslint && if [ -x ""/Users/mzasso/git/nodejs/node/node"" ] && [ -e ""/Users/mzasso/git/nodejs/node/node"" ]; then ""/Users/mzasso/git/nodejs/node/node"" /Users/mzasso/git/nodejs/node/./deps/npm/bin/npm-cli.js ci; elif [ -x `command -v node` ] && [ -e `command -v node` ] && [ `command -v node` ]; then `command -v node` /Users/mzasso/git/nodejs/node/./deps/npm/bin/npm-cli.js ci; else echo "No available node, cannot run \"node /Users/mzasso/git/nodejs/node/./deps/npm/bin/npm-cli.js ci\""; exit 1; fi;
npm warn cli npm v10.8.3 does not support Node.js v23.0.0-pre. This version of npm supports the following node versions: `^18.17.0 || >=20.5.0`. You can find the latest version at https://nodejs.org/.
(node:45192) ExperimentalWarning: Support for loading ES Module in require() is an experimental feature and might change at any time
    at emitExperimentalWarning (node:internal/util:269:11)
    at loadESMFromCJS (node:internal/modules/cjs/loader:1388:5)
    at Module._compile (node:internal/modules/cjs/loader:1517:5)
    at Module._extensions..js (node:internal/modules/cjs/loader:1672:16)
    at Module.load (node:internal/modules/cjs/loader:1328:32)
    at Module._load (node:internal/modules/cjs/loader:1138:12)
    at TracingChannel.traceSync (node:diagnostics_channel:315:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:218:24)
    at Module.require (node:internal/modules/cjs/loader:1350:12)
    at require (node:internal/modules/helpers:138:16)

added 185 packages, and audited 186 packages in 8s

41 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
Running JS linter...

joyeecheung · 2024-10-05T20:24:06Z

Yes, because of debug, which I covered in #55217 (comment) and addressed in the last commit of the backport PR. In the last TSC meeting we agreed to silence the warning when require() comes from node_modules on v22 and keep it on v23 to help people notice and update them (with #55241)

This unflags --experimental-require-module so require(esm) can be used without the flag. For now, when require() actually encounters an ESM, it will still emit an experimental warning. To opt out of the feature, --no-experimental-require-module can be used. There are some tests specifically testing ERR_REQUIRE_ESM. Some of them are repurposed to test --no-experimental-require-module. Some of them are modified to just expect loading require(esm) to work, when it's appropriate. PR-URL: nodejs#55085 Refs: nodejs#52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: LiviaMedeiros <livia@cirno.name> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Filip Skokan <panva.ip@gmail.com> Reviewed-By: Michael Dawson <midawson@redhat.com> Reviewed-By: Richard Lau <rlau@redhat.com>

When a ESM module cannot be loaded by require due to the presence of TLA, its module status would be stopped at kInstantiated. In this case, when it's imported again, we should allow it to be evaluated asynchronously, as it's also a common pattern for users to retry with dynamic import when require fails. PR-URL: #55502 Fixes: #55500 Refs: #52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chemi Atlow <chemi@atlow.co.il>

This tracks the asynchronicity in the ModuleWraps when they turn out to contain TLA after instantiation, and throw the right error (ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes the freezing of ModuleWraps since it's not meaningful to freeze this when the rest of the module loader is mutable, and we can record the asynchronicity in the ModuleWrap right after compilation after we get a V8 upgrade that contains v8::Module::HasTopLevelAwait() instead of searching through the module graph repeatedly which can be slow. PR-URL: #55520 Fixes: #55516 Refs: #52697 Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com> Reviewed-By: Jacob Smith <jacob@frende.me>

This patch implements a "module-sync" exports condition for packages to supply a sycnrhonous ES module to the Node.js module loader, no matter it's being required or imported. This is similar to the "module" condition that bundlers have been using to support `require(esm)` in Node.js, and allows dual-package authors to opt into ESM-first only newer versions of Node.js that supports require(esm) while avoiding the dual-package hazard. ```json { "type": "module", "exports": { "node": { // On new version of Node.js, both require() and import get // the ESM version "module-sync": "./index.js", // On older version of Node.js, where "module" and // require(esm) are not supported, use the transpiled CJS version // to avoid dual-package hazard. Library authors can decide // to drop support for older versions of Node.js when they think // it's time. "default": "./dist/index.cjs" }, // On any other environment, use the ESM version. "default": "./index.js" } } ``` We end up implementing a condition with a different name instead of reusing "module", because existing code in the ecosystem using the "module" condition sometimes also expect the module resolution for these ESM files to work in CJS style, which is supported by bundlers, but the native Node.js loader has intentionally made ESM resolution different from CJS resolution (e.g. forbidding `import './noext'` or `import './directory'`), so it would be semver-major to implement a `"module"` condition without implementing the forbidden ESM resolution rules. For now, this just implments a new condition as semver-minor so it can be backported to older LTS. Refs: https://webpack.js.org/guides/package-exports/#target-environment-independent-packages PR-URL: nodejs#54648 Fixes: nodejs#52173 Refs: https://github.com/joyeecheung/test-module-condition Refs: nodejs#52697 Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Jan Krems <jan.krems@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com>

This unflags --experimental-require-module so require(esm) can be used without the flag. For now, when require() actually encounters an ESM, it will still emit an experimental warning. To opt out of the feature, --no-experimental-require-module can be used. There are some tests specifically testing ERR_REQUIRE_ESM. Some of them are repurposed to test --no-experimental-require-module. Some of them are modified to just expect loading require(esm) to work, when it's appropriate. PR-URL: nodejs#55085 Refs: nodejs#52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: LiviaMedeiros <livia@cirno.name> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Filip Skokan <panva.ip@gmail.com> Reviewed-By: Michael Dawson <midawson@redhat.com> Reviewed-By: Richard Lau <rlau@redhat.com>

When a ESM module cannot be loaded by require due to the presence of TLA, its module status would be stopped at kInstantiated. In this case, when it's imported again, we should allow it to be evaluated asynchronously, as it's also a common pattern for users to retry with dynamic import when require fails. PR-URL: nodejs#55502 Fixes: nodejs#55500 Refs: nodejs#52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chemi Atlow <chemi@atlow.co.il>

This tracks the asynchronicity in the ModuleWraps when they turn out to contain TLA after instantiation, and throw the right error (ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes the freezing of ModuleWraps since it's not meaningful to freeze this when the rest of the module loader is mutable, and we can record the asynchronicity in the ModuleWrap right after compilation after we get a V8 upgrade that contains v8::Module::HasTopLevelAwait() instead of searching through the module graph repeatedly which can be slow. PR-URL: nodejs#55520 Fixes: nodejs#55516 Refs: nodejs#52697 Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com> Reviewed-By: Jacob Smith <jacob@frende.me>

This unflags --experimental-require-module so require(esm) can be used without the flag. For now, when require() actually encounters an ESM, it will still emit an experimental warning. To opt out of the feature, --no-experimental-require-module can be used. There are some tests specifically testing ERR_REQUIRE_ESM. Some of them are repurposed to test --no-experimental-require-module. Some of them are modified to just expect loading require(esm) to work, when it's appropriate. PR-URL: nodejs#55085 Backport-PR-URL: nodejs#55217 Refs: nodejs#52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: LiviaMedeiros <livia@cirno.name> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Filip Skokan <panva.ip@gmail.com> Reviewed-By: Michael Dawson <midawson@redhat.com> Reviewed-By: Richard Lau <rlau@redhat.com>

This is faster and more consistent with other places using the regular expression to detect node_modules. PR-URL: nodejs#55243 Backport-PR-URL: nodejs#55217 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Refs: nodejs#52697

Some packages have been using try-catch to load require(esm) in environments that are available. In 23, where require(esm) is unflagged, we emit an experimental warning when require() is used to load ESM. To backport require(esm) to older LTS releases, however, this could break existing CLI output. To reduce the disruption for LTS, on older release lines, this commit is applied to skip the warning if require(esm) comes from node_modules. This warning will be eventually removed when require(esm) becomes stable. PR-URL: nodejs#55217 Refs: nodejs#52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

This unflags --experimental-require-module so require(esm) can be used without the flag. For now, when require() actually encounters an ESM, it will still emit an experimental warning. To opt out of the feature, --no-experimental-require-module can be used. There are some tests specifically testing ERR_REQUIRE_ESM. Some of them are repurposed to test --no-experimental-require-module. Some of them are modified to just expect loading require(esm) to work, when it's appropriate. PR-URL: nodejs#55085 Backport-PR-URL: nodejs#55217 Refs: nodejs#52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: LiviaMedeiros <livia@cirno.name> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Filip Skokan <panva.ip@gmail.com> Reviewed-By: Michael Dawson <midawson@redhat.com> Reviewed-By: Richard Lau <rlau@redhat.com>

This is faster and more consistent with other places using the regular expression to detect node_modules. PR-URL: nodejs#55243 Backport-PR-URL: nodejs#55217 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Refs: nodejs#52697

Some packages have been using try-catch to load require(esm) in environments that are available. In 23, where require(esm) is unflagged, we emit an experimental warning when require() is used to load ESM. To backport require(esm) to older LTS releases, however, this could break existing CLI output. To reduce the disruption for LTS, on older release lines, this commit is applied to skip the warning if require(esm) comes from node_modules. This warning will be eventually removed when require(esm) becomes stable. PR-URL: nodejs#55217 Refs: nodejs#52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

When a ESM module cannot be loaded by require due to the presence of TLA, its module status would be stopped at kInstantiated. In this case, when it's imported again, we should allow it to be evaluated asynchronously, as it's also a common pattern for users to retry with dynamic import when require fails. PR-URL: nodejs#55502 Fixes: nodejs#55500 Refs: nodejs#52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chemi Atlow <chemi@atlow.co.il>

This is faster and more consistent with other places using the regular expression to detect node_modules. PR-URL: nodejs#55243 Backport-PR-URL: nodejs#55217 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Refs: nodejs#52697

Some packages have been using try-catch to load require(esm) in environments that are available. In 23, where require(esm) is unflagged, we emit an experimental warning when require() is used to load ESM. To backport require(esm) to older LTS releases, however, this could break existing CLI output. To reduce the disruption for LTS, on older release lines, this commit is applied to skip the warning if require(esm) comes from node_modules. This warning will be eventually removed when require(esm) becomes stable. PR-URL: nodejs#55217 Refs: nodejs#52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

When a ESM module cannot be loaded by require due to the presence of TLA, its module status would be stopped at kInstantiated. In this case, when it's imported again, we should allow it to be evaluated asynchronously, as it's also a common pattern for users to retry with dynamic import when require fails. PR-URL: nodejs#55502 Fixes: nodejs#55500 Refs: nodejs#52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chemi Atlow <chemi@atlow.co.il>

jonkoops · 2024-11-20T19:37:29Z

@joyeecheung is the intention to eventually backport your efforts to 20.x and 18.x as well? Or is your intention to land this in 22.x and 23.x only?

joyeecheung · 2024-11-20T21:29:06Z

From the release WG session at the collab summit, we had the greenlight to unflag it to v20.x, assuming that it is backportable - v20.x is a pretty old branch compared to main so it will take some effort to backport all the recent changes, which involve some refactoring. It will also only happen after v22.x unflagging is released and tested. If things go well in v22.x, and we manage to backport it properly to v20.x, we'll unflag it in v20.x in a later release.

jonkoops · 2024-11-20T22:04:20Z

Thanks for the explainer. I assume that 18.x will not be receiving any backports considering its relative old age and EOL next year?

joyeecheung · 2024-11-21T07:37:24Z

Yeah v18.x is not impossible, but it's up to the release team to decide. To backport it we'll need

The release team giving it a green light
A volunteer doing the backport (finding and git-cherry-picking all the relevant patches from main to v18.x-staging, resolving all the conflicts and making sure it's still working as intended, then opening a backport PR to v18.x-staging branch)
A volunteer from the releasers team to merge the backport PR to v18.x-staging and do all the steps in https://github.com/nodejs/node/blob/main/doc/contributing/releases.md to complete the release.

Not sure about 1 because it was never really discussed. 2-3 will probably lack volunteers considering v18.x is approaching EOL (2 can be done by anyone interested in doing so, doesn't need to be an existing contributor though).

This patch implements a "module-sync" exports condition for packages to supply a sycnrhonous ES module to the Node.js module loader, no matter it's being required or imported. This is similar to the "module" condition that bundlers have been using to support `require(esm)` in Node.js, and allows dual-package authors to opt into ESM-first only newer versions of Node.js that supports require(esm) while avoiding the dual-package hazard. ```json { "type": "module", "exports": { "node": { // On new version of Node.js, both require() and import get // the ESM version "module-sync": "./index.js", // On older version of Node.js, where "module" and // require(esm) are not supported, use the transpiled CJS version // to avoid dual-package hazard. Library authors can decide // to drop support for older versions of Node.js when they think // it's time. "default": "./dist/index.cjs" }, // On any other environment, use the ESM version. "default": "./index.js" } } ``` We end up implementing a condition with a different name instead of reusing "module", because existing code in the ecosystem using the "module" condition sometimes also expect the module resolution for these ESM files to work in CJS style, which is supported by bundlers, but the native Node.js loader has intentionally made ESM resolution different from CJS resolution (e.g. forbidding `import './noext'` or `import './directory'`), so it would be semver-major to implement a `"module"` condition without implementing the forbidden ESM resolution rules. For now, this just implments a new condition as semver-minor so it can be backported to older LTS. Refs: https://webpack.js.org/guides/package-exports/#target-environment-independent-packages PR-URL: nodejs#54648 Fixes: nodejs#52173 Refs: https://github.com/joyeecheung/test-module-condition Refs: nodejs#52697 Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Jan Krems <jan.krems@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com>

This unflags --experimental-require-module so require(esm) can be used without the flag. For now, when require() actually encounters an ESM, it will still emit an experimental warning. To opt out of the feature, --no-experimental-require-module can be used. There are some tests specifically testing ERR_REQUIRE_ESM. Some of them are repurposed to test --no-experimental-require-module. Some of them are modified to just expect loading require(esm) to work, when it's appropriate. PR-URL: nodejs#55085 Refs: nodejs#52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: LiviaMedeiros <livia@cirno.name> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Filip Skokan <panva.ip@gmail.com> Reviewed-By: Michael Dawson <midawson@redhat.com> Reviewed-By: Richard Lau <rlau@redhat.com>

When a ESM module cannot be loaded by require due to the presence of TLA, its module status would be stopped at kInstantiated. In this case, when it's imported again, we should allow it to be evaluated asynchronously, as it's also a common pattern for users to retry with dynamic import when require fails. PR-URL: nodejs#55502 Fixes: nodejs#55500 Refs: nodejs#52697 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Chemi Atlow <chemi@atlow.co.il>

This tracks the asynchronicity in the ModuleWraps when they turn out to contain TLA after instantiation, and throw the right error (ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes the freezing of ModuleWraps since it's not meaningful to freeze this when the rest of the module loader is mutable, and we can record the asynchronicity in the ModuleWrap right after compilation after we get a V8 upgrade that contains v8::Module::HasTopLevelAwait() instead of searching through the module graph repeatedly which can be slow. PR-URL: nodejs#55520 Fixes: nodejs#55516 Refs: nodejs#52697 Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Chengzhong Wu <legendecas@gmail.com> Reviewed-By: Jacob Smith <jacob@frende.me>

jonkoops · 2024-11-21T10:56:18Z

I'd volunteer to help out to get the backport for v18.x done, as I find your work incredibly valuable, especially as a package author dealing with the module dichotomy. But I fear that I would lack the required expertise (and time) to get it done. Still, it would be incredibly valuable to the larger JavaScript community.

voxpelli · 2024-11-21T11:24:29Z

There’s no feature freeze or such a few months prior to a major losing LTS status? Adding something new that can introduce bugs right before stopping all bug fixing feels wrong

joyeecheung · 2024-11-21T11:46:06Z

I don't think there is a feature freeze policy, but things are subject to the discretion of the release team (hence 1 mentioned above)

richardlau · 2024-11-21T13:24:40Z

Node.js 18 is in the maintenance phase of LTS and the default position of the Release WG is that new features do not land on it:

https://github.com/nodejs/Release?tab=readme-ov-file#release-phases

Maintenance - Critical bug fixes and security updates. New features may be added at the discretion of the Release team - typically only in cases where the new feature supports migration to later release lines.

Now "supports migration to later release lines" could reasonably be applied to this feature, but it would have to be balanced against risk and personally I'd need a lot more convincing to allow it on Node.js 18 given differences in module loading (several things that are on main/23 are not in Node.js 18 -- if there's uncertainty/trickiness in backporting to 20, backporting to 18 is going to be even harder/riskier).

FWIW the discretion part was added for smaller, self-contained things such as node-addon-api features.

voxpelli · 2024-11-21T15:52:25Z

If things go well in v22.x, and we manage to backport it properly to v20.x, we'll unflag it in v20.x in a later release.

And that timeline gives very little time to land anything in v18.x, as it should first be proven in v22.x, then added to v20.x and probably become proven there as well, and only after that would it be up for backport into v18.x, and who knows at which date that ends up, probably at least January or February?

joyeecheung added module Issues and PRs related to the module subsystem. esm Issues and PRs related to the ECMAScript Modules implementation. labels Apr 25, 2024

RyanCavanaugh mentioned this issue Apr 29, 2024

Default import from an ESM package into a CommonJS project with esModuleInterop is faulty microsoft/TypeScript#58341

Closed

GeoffreyBooth mentioned this issue May 3, 2024

deps: vendor emphasize #52826

Closed

dgozman mentioned this issue Aug 8, 2024

[Feature] Add a way to handle CJS/ESM module hell microsoft/playwright#23662

Closed

joyeecheung mentioned this issue Aug 20, 2024

Session Proposal: module loading customization/optimization and CJS/ESM interoperability openjs-foundation/summit#422

Closed

6 tasks

joyeecheung mentioned this issue Aug 29, 2024

module: implement the "module-sync" exports condition #54648

Merged

This was referenced Sep 19, 2024

Node.js 23 release plan nodejs/Release#1034

Closed

module: unflag --experimental-require-module #55085

Merged

voxpelli mentioned this issue Oct 4, 2024

Bug: no-missing-import doesn't use the require algorithm when TS has CJS moduleResolution? eslint-community/eslint-plugin-n#349

Closed

1 task

joyeecheung mentioned this issue Oct 5, 2024

[v22.x] backport unflagging of --experimental-require-module and related fixes/refactorings #55217

Open

tats-u mentioned this issue Oct 8, 2024

Enable consuming of ES modules in extensions microsoft/vscode#130367

Open

jonkoops mentioned this issue Nov 20, 2024

debug causes npm to always issue an experimental ESM warning in node 23 debug-js/debug#975

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tracking issue: require(esm) #52697

Tracking issue: require(esm) #52697

joyeecheung commented Apr 25, 2024 •

edited

Loading

GeoffreyBooth commented Apr 30, 2024

jakebailey commented May 7, 2024

Andarist commented May 7, 2024

joyeecheung commented Jul 11, 2024 •

edited

Loading

GeoffreyBooth commented Jul 11, 2024

joyeecheung commented Aug 29, 2024 •

edited

Loading

voxpelli commented Oct 3, 2024

voxpelli commented Oct 4, 2024

targos commented Oct 5, 2024

joyeecheung commented Oct 5, 2024 •

edited

Loading

jonkoops commented Nov 20, 2024

joyeecheung commented Nov 20, 2024

jonkoops commented Nov 20, 2024

joyeecheung commented Nov 21, 2024 •

edited

Loading

jonkoops commented Nov 21, 2024

voxpelli commented Nov 21, 2024

joyeecheung commented Nov 21, 2024

richardlau commented Nov 21, 2024

voxpelli commented Nov 21, 2024

Tracking issue: require(esm) #52697

Tracking issue: require(esm) #52697

Comments

joyeecheung commented Apr 25, 2024 • edited Loading

GeoffreyBooth commented Apr 30, 2024

jakebailey commented May 7, 2024

Andarist commented May 7, 2024

joyeecheung commented Jul 11, 2024 • edited Loading

GeoffreyBooth commented Jul 11, 2024

joyeecheung commented Aug 29, 2024 • edited Loading

voxpelli commented Oct 3, 2024

voxpelli commented Oct 4, 2024

targos commented Oct 5, 2024

joyeecheung commented Oct 5, 2024 • edited Loading

jonkoops commented Nov 20, 2024

joyeecheung commented Nov 20, 2024

jonkoops commented Nov 20, 2024

joyeecheung commented Nov 21, 2024 • edited Loading

jonkoops commented Nov 21, 2024

voxpelli commented Nov 21, 2024

joyeecheung commented Nov 21, 2024

richardlau commented Nov 21, 2024

voxpelli commented Nov 21, 2024

joyeecheung commented Apr 25, 2024 •

edited

Loading

joyeecheung commented Jul 11, 2024 •

edited

Loading

joyeecheung commented Aug 29, 2024 •

edited

Loading

joyeecheung commented Oct 5, 2024 •

edited

Loading

joyeecheung commented Nov 21, 2024 •

edited

Loading