-
Notifications
You must be signed in to change notification settings - Fork 29.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
esm: do not call getSource
when format is commonjs
#50465
Conversation
Review requested:
|
If the observable output is changing then that’s a breaking change, so it’s pretty important to fix. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! A few comments regarding the tests:
I feel like the bug will just resurface whenever custom hooks are present and returning a |
Thanks @GeoffreyBooth, I've created a new branch with the change you suggested. I had to use a new internal property I like this change because it replaces a synchronous read operation with an asynchronous one. The downside is that there would be a discrepancy between |
We cannot use the source from the ESM loader, it would short cut all the monkey patching of the CJS loader. |
It doesn't if |
I think the actual fix would be to ask the main thread CJS loader for the source rather than calling |
That's a bummer because we are looking for ways to remove dependency on the monkey patchable CJS loader, but oh well. Or we accept the breaking change, and ask folks who rely on the monkey patchability of the CJS loader to switch to the new API (which mean we can't unflag until the ecosystem is ready) |
Because monkey patching is unsupported, there's no defined API surface. We don't really know what would or wouldn't break people. In this case, I think the only impact would be people who monkey patch the CJS loader in order to affect |
Which is everyone you mean? I think Yarn PnP relies on that for example, or any project that uses |
No, that's relying on monkey patching to affect |
I don’t know what makes you think that, I think that’s not correct. |
Can you provide an example where someone is monkey-patching the CommonJS loader to affect Regardless, I don’t think it’s a supported use case. I don’t think monkey-patching at all is a supported use case, and monkey-patching the CommonJS loader to affect only CommonJS files in a mixed CommonJS-ESM codebase is even less supported: we provide the module hooks for that. |
Do you mean examples other than the one I gave in #50465 (comment)? Could you clarify why you think those are not good examples?
not really, if you provide a |
(nb, also |
Since this PR is all about avoiding to read the same file twice with the ESM loader, maybe it would be better to continue the discussion about the CommonJS monkey-patching legacy in a separate issue to give it more visibility and solicit other perspectives. It looks like there is more need to speak about that. Meanwhile, I'm happy to update and extend the unit tests as suggested so we can move on with this issue. |
@GeoffreyBooth I've added a unit test to ensure that the file specified in the URL is not used when a |
import assert from 'assert'; | ||
|
||
const { default: existingFileSource } = await import(fixtures.fileURL('es-modules', 'cjs-file.cjs')); | ||
const { default: noSuchFileSource } = await import(fixtures.fileURL('no-such-file.cjs')); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: since this does not in fact reference a fixtures, let's simplify.
const { default: noSuchFileSource } = await import(fixtures.fileURL('no-such-file.cjs')); | |
const { default: noSuchFileSource } = await import('./no-such-file.cjs'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Unfortunately, './no-such-file.cjs'
wouldn't work because it's not a valid URL, and the resolve
hook doesn't actually resolve relative paths because of the short-circuit. Anyway, a file URL like 'file:///no-such-file.cjs'
works fine and is not relative (at least on Unix), so I switched to use that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's incorrect to say it's not a valid URL, it is a valid (relative) URL.
Anyway, a file URL like
'file:///no-such-file.cjs'
works fine
IMO it would make more sense to use a relative URL in the JS, and have the hook return a specific URL. That doesn't really matter anyway, for the purpose of this test the current version works, so good enough for me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Never mind, I had to change the test again because Windows doesn't like the absolute file URL without a drive letter: 4dcbec5. Let's see how the CI build goes this time.
Landed in 2602c46 |
Ensure that `defaultLoad` does not uselessly access the file system to get the source of modules that are known to be in CommonJS format. This allows CommonJS imports to resolve in the current phase of the event loop. Refs: eslint/eslint#17683 PR-URL: #50465 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Ensure that `defaultLoad` does not uselessly access the file system to get the source of modules that are known to be in CommonJS format. This allows CommonJS imports to resolve in the current phase of the event loop. Refs: eslint/eslint#17683 PR-URL: #50465 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Ensure that `defaultLoad` does not uselessly access the file system to get the source of modules that are known to be in CommonJS format. This allows CommonJS imports to resolve in the current phase of the event loop. Refs: eslint/eslint#17683 PR-URL: #50465 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Ensure that `defaultLoad` does not uselessly access the file system to get the source of modules that are known to be in CommonJS format. This allows CommonJS imports to resolve in the current phase of the event loop. Refs: eslint/eslint#17683 PR-URL: #50465 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Ensure that `defaultLoad` does not uselessly access the file system to get the source of modules that are known to be in CommonJS format. This allows CommonJS imports to resolve in the current phase of the event loop. Refs: eslint/eslint#17683 PR-URL: #50465 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
* chore: bump node in DEPS to v20.11.0 * module: bootstrap module loaders in shadow realm nodejs/node#48655 * src: add commit hash shorthand in zlib version nodejs/node#50158 * v8,tools: expose necessary V8 defines nodejs/node#50820 * esm: do not call getSource when format is commonjs nodejs/node#50465 * esm: fallback to readFileSync when source is nullish nodejs/node#50825 * vm: allow dynamic import with a referrer realm nodejs/node#50360 * test: skip test-diagnostics-channel-memory-leak.js nodejs/node#50327 * esm: do not call getSource when format is commonjs nodejs/node#50465 * lib: fix assert throwing different error messages in ESM and CJS nodejs/node#50634 * src: fix compatility with upcoming V8 12.1 APIs nodejs/node#50709 * deps: update base64 to 0.5.1 nodejs/node#50629 * src: avoid silent coercion to signed/unsigned int nodejs/node#50663 * src: fix compatility with upcoming V8 12.1 APIs nodejs/node#50709 * chore: fix patch indices * chore: update patches * test: disable TLS cipher test This can't be enabled owing to BoringSSL incompatibilities. nodejs/node#50186 * fix: check for Buffer and global definition in shadow realm nodejs/node#51239 * test: disable parallel/test-shadow-realm-custom-loader Incompatible with our asar logic, resulting in the following failure: > Failed to CompileAndCall electron script: electron/js2c/asar_bundle * chore: remove deleted parallel/test-crypto-modp1-error test * test: make test-node-output-v8-warning generic nodejs/node#50421 * chore: fixup ModuleWrap patch * test: match wpt/streams/transferable/transform-stream-members.any.js to upstream * fix: sandbox is not enabled on arm * chore: disable v8 sandbox on ia32/arm --------- Co-authored-by: electron-roller[bot] <84116207+electron-roller[bot]@users.noreply.github.com> Co-authored-by: Shelley Vohr <shelley.vohr@gmail.com> Co-authored-by: Cheng Zhao <zcbenz@gmail.com>
This PR ensures that
defaultLoad
does not access the file system to read the source of modules that are known in advance to be in CommonJS format. This is not necessary, since the CommonJS loader can't use the source property and must instead read the file once again.For context, this was noticed while investigating a test failure in ESLint on Node.js 21.1.0. Our logic (now fixed) was incorrectly assuming that some asynchronous operations would complete in a specific order, and this order has changed from Node.js 21.0.0 to 21.1.0.
The simplest repro I've found is this one:
with
file1.cjs
andfile2.cjs
both empty modules.This is the output of running
index.mjs
in Node.js 21.0.0:And this is the output in Node.js 21.1.0:
Here is another similar repro in a single file:
The reason for the inverted order is this commit and a later change, which is causing
getSource
to be called even when the format of a module is already known to becommonjs
, for example because of the.cjs
extension.getSource
accesses the file system and that consistently requires one or more I/O cycles to complete. This allows callbacks scheduled withprocess.nextTick
to run in the meantime.Clearly, importing a CommonJS module is an asynchronous operation which souldn't be assumed to resolve in a specific order or number of phases, but calling
getSource
for known CommonJS modules isn't needed and should be avoided to not occupy the event loop unnecessarily.Refs: eslint/eslint#17683