Prefix node built-in module imports with node:.

[lgarron]

This makes the imports distinguishable from the corresponding npm packages, which can help with:

• Development ergonomics:
  • Marking all node:* packages as external in a bundler (or even better, enabling them to do so automatically).
  • Analyzing a project's dependency graph.
• Supply chain security (avoiding risks from name squatting).

The node: protocol prefix is supported in LTS versions all the way back to node 12: https://nodejs.org/api/esm.html#node-imports

[lgarron]

Responding to #17915 (comment) by @RReverser

I think node: prefix is a relatively recent addition in Node 18+, and not something supported in Node 14 that Emscripten ships with. Besides, I'm not sure what the bundler & other tooling support is like.

I'd rather err on the side of compatibility with more tooling for now.

Given the link above documenting that this is supported in node LTS back to v12, does that change your mind?

I regularly work with code that is difficult or impossible to polyfill in older versions of node. It's important for me not to be held behind by older versions, especially when it concerns code correctness and security (as is the case here), or when it comes time to put replace incompatible standards in the rearview mirror (as is also the case here).

If this is not quite appropriate to do by default at this time, would a flag be appropriate to emit such prefixed imports?

[RReverser]

Given the link above documenting that this is supported in node LTS back to v12, does that change your mind?

Yeah I was basing my assumption off articles like this - https://fusebit.io/blog/node-18-prefix-only-modules/ - and my own experiencing seeing node: prefix only very recently. If it's supported all the way back, great.

What about my other part - does it work as expected with bundlers e.g. webpack / rollup?

[RReverser]

It's important for me not to be held behind by older versions

As a general point, sure, I get that might be the requirement for a specific project, just saying that at the tooling level backward compat is often more important as tooling like Emscripten needs to support a lot of projects in the wild.

But then again, since this feature is supported all the way back to Node 12, this point is moot here :)

[lgarron]

It's important for me not to be held behind by older versions

As a general point, sure, I get that might be the requirement for a specific project, just saying that at the tooling level backward compat is often more important as tooling like Emscripten needs to support a lot of projects in the wild.

Yeah, I understand, supporting both modern and older targets can certainly be very time-consuming, especially when node is concerned. 😬

For some context, I've spent a lot of time recently trying to get a C++ project to build to a usable WASM output using a variety of WASM tools. My first impression of Emscripten's emitted wrapper file was that it had a lot of outdated assumptions and bulky workaround code that we would never need, and that it would be a pain to get working in node. I've been putting off trying to find a way to address this, until I was pleasantly surprised to see that tip-of-tree now works out of the box.

This PR is my attempt to address the main remaining artifact of the current code that affects my day-to-day work, but it's definitely a much smaller detail! It would definitely be nice to ask em++ to handle this, though, rather than try to rewrite the output on my end.

[RReverser]

Yeah, as I said, personally I think it's reasonable given that my understanding of this being a bleeding edge feature was wrong, just want to double-check

does it work as expected with bundlers e.g. webpack / rollup?

[kleisauke]

Given the link above documenting that this is supported in node LTS back to v12, does that change your mind?

This is not quite true, the node: prefix in require(...) is only supported since v14.18.0. For example:

Details

$ nvm install 12
$ nvm use 12
Now using node v12.22.12 (npm v6.14.16)
$ node -v
v12.22.12
$ node -e 'require("node:fs").writeSync(1, "Hello world!\n");'
internal/modules/cjs/loader.js:818
  throw err;
  ^

Error: Cannot find module 'node:fs'
$ node -e 'require("fs").writeSync(1, "Hello world!\n");'
Hello world!
$ nvm install 14.17
$ nvm use 14.17
Now using node v14.17.6 (npm v6.14.15)
$ node -v
v14.17.6
$ node -e 'require("node:fs").writeSync(1, "Hello world!\n");'
internal/modules/cjs/loader.js:892
  throw err;
  ^

Error: Cannot find module 'node:fs'
$ node -e 'require("fs").writeSync(1, "Hello world!\n");'
Hello world!
$ nvm install 14.18.0
$ nvm use 14.18.0
Now using node v14.18.0 (npm v6.14.15)
$ node -v
v14.18.0
$ node -e 'require("node:fs").writeSync(1, "Hello world!\n");'
Hello world!

We should probally add a MIN_NODEJS_VERSION option first (just like MIN_CHROME_VERSION, MIN_FIREFOX_VERSION, etc.) that allows the use of this when specifying -sMIN_NODEJS_VERSION=14.18.

Adding such an option would also mean that we could set -sNODEJS_CATCH_EXIT=0 -sNODEJS_CATCH_REJECTION=0 automatically (when doing -sMIN_NODEJS_VERSION=15.0) or remove the use of certain pollyfills at compile-time.

emscripten/src/shell.js

Lines 409 to 415 in 7307f41

	// Polyfill the performance object, which emscripten pthreads support
	// depends on for good timing.
	// Note: this is no longer needed on Node.js >= 16, see:
	// https://nodejs.org/api/globals.html#performance
	if (typeof performance == 'undefined') {
	global.performance = require('perf_hooks').performance;
	}

(i.e., the above could be removed when doing -sMIN_NODEJS_VERSION=16.0)

[lgarron]

This is not quite true, the node: prefix in require(...) is only supported since v14.18.0.

Hmm, I was going off the official docs, that's a little concerning. 😬

just want to double-check

does it work as expected with bundlers e.g. webpack / rollup?

I don't use Webpack or Rollup anymore, so unfortunately I can't help answer that. 😕

We should probally add a MIN_NODEJS_VERSION option first (just like MIN_CHROME_VERSION, MIN_FIREFOX_VERSION, etc.) that allows the use of this when specifying -sMIN_NODEJS_VERSION=14.18.

I'd be very happy with that! 😃

Many projects specify an "engines" field in their package.json, and therefore already know what version they're targeting for compatibility: https://docs.npmjs.com/cli/v9/configuring-npm/package-json#engines
So this would be a fairly straightforward option to select for such projects.

[kripken]

Adding MIN_NODEJS_VERSION might make sense. Another option might be to use runtime checking, as code size in node.js builds is not as critical as Web builds? That is, at runtime we could do a different import in newer versions of node. A benefit to doing it that way is that people wouldn't need to set the flag at link time to get the benefit.

[lgarron]

Another option might be to use runtime checking, as code size in node.js builds is not as critical as Web builds? That is, at runtime we could do a different import in newer versions of node.

As someone who works hard to write portable code, I would like to strongly argue against this. It's hard enough providing one universally compatible build with a slimmed-down set of workarounds, and I cannot afford to maintain multiple builds to keep the code size down.

In addition, it can often be a bad idea to leave unused code around for a long time, as bundlers or browsers will start having issues with code that isn't ever actually going to run, which can waste a whole bunch of debugging time. 😕
(Not necessarily because I want to use those tools, but because others expect my projects to run through their preferred bundler without issues.)

[kripken]

@lgarron Sounds reasonable. Ok, if the general preference is MIN_NODEJS_VERSION then let's go with that.

[lgarron]

poke 🤓

[sbc100]

Given that emsdk uses node v14.18.2 I'd be in favor of simply declaring that we don't support node versions older than that. I'm also find with landing this change as-is without blocking on adding the new MIN_NODEJS_VERSION setting. What do others think?

[kripken]

Do we have a sense of whether users are running the output on older versions? I suggest we at least ask on the mailing list first. If there are no concerns there then I'm also ok with landing this as-is, maybe with docs that mention the minimum node required.

[lgarron]

I think this would still be great to land. I've rebased and updated the PR.

(Note: There are a still bunch of calls in test code that I haven't updated, in order to keep the PR as simple as possible.)

[sbc100]

This will regress code slightly for all targets, right?

Given that, I wonder if it is worth it? Regarding npm, are there really packages that conflict in their naming with the set of modules that we require? Are bundlers really confused by the lack of clarity? Are there actual negative outcomes to this confusion?

[lgarron]

This will regress code slightly for all targets, right?

The code will have exactly the same functionality.

As discussed above, this is supported in node LTS a long while back (as well as in other runtimes).

Given that, I wonder if it is worth it? Regarding npm, are there really packages that conflict in their naming with the set of modules that we require?

Yes.

I have had some gnarly times fighting very bad assumptions in the ecosystem about the availability and functionality of the events library vs. the node built-in.

Are bundlers really confused by the lack of clarity?

They're not so much confused as they have to make assumptions that cannot be correct under all circumstances. There is literally no way for them to know which import is actually meant, whereas a node: prefix makes it super clear.

Are there actual negative outcomes to this confusion?

Yes. It can be difficult or impossible ship code that works properly in all bundlers when the node: prefix is not available.

• As mentioned in the PR comment, this messes with code analysis and semantics.
• If you acquire a dependency with the same node built-in module then the semantics for the import will change, and not always in predictable ways (i.e. different bundlers and runtimes might end up with different interpretations). Some bundlers also try to inject polyfills under some conditions, making this even more complicated. And worst of all, this can happen even if you personally did not add a dependency with the same name as a node built-in module (e.g. if one of your dependencies introduced it as a transitive dependency).
• It's much easier to tell/expect your downstream users to ignore imports with a node: prefix when analyzing or bundling code. Otherwise, every newly imported node built-in in your dependency graph can be a breaking change for consuming code.

All of this adds up to a few footguns for "normal" projects. But more importantly, if you're a library author publishing code with WASM, you have to deal with any semantic issues across all runtimes and bundlers used by people using your library.

If it's any indication of how much a difference this makes to me as a library author, I use scripts to always rewrite Emscripten output to add the node: prefix to imports because it reduces the ecosystem issues described above.

[sbc100]

This will regress code slightly for all targets, right?

The code will have exactly the same functionality.

I mean it will regress code size slightly. Each program we generate will get a few bytes bigger.

Given that, I wonder if it is worth it? Regarding npm, are there really packages that conflict in their naming with the set of modules that we require?

Yes.

I have had some gnarly times fighting very bad assumptions in the ecosystem about the availability and functionality of the events library vs. the node built-in.

Yes but is that try for any of the module that we use here in emscripten?

Yes. It can be difficult or impossible ship code that works properly in all bundlers when the node: prefix is not available.

• As mentioned in the PR comment, this messes with code analysis and semantics.
• If you acquire a dependency with the same node built-in module then the semantics for the import will change, and not always in predictable ways (i.e. different bundlers and runtimes might end up with different interpretations). Some bundlers also try to inject polyfills under some conditions, making this even more complicated. And worst of all, this can happen even if you personally did not add a dependency with the same name as a node built-in module (e.g. if one of your dependencies introduced it as a transitive dependency).
• It's much easier to tell/expect your downstream users to ignore imports with a node: prefix when analyzing or bundling code. Otherwise, every newly imported node built-in in your dependency graph can be a breaking change for consuming code.

All of this adds up to a few footguns for "normal" projects. But more importantly, if you're a library author publishing code with WASM, you have to deal with any semantic issues across all runtimes and bundlers used by people using your library.

If it's any indication of how much a difference this makes to me as a library author, I use scripts to always rewrite Emscripten output to add the node: prefix to imports because it reduces the ecosystem issues described above.

That does sound like it can be quite bad. Can you point to bundler where this true, or an example with will fail with the current emscripten output? We have tests that run under rollup, vite and webpack:

emscripten/test/test_browser.py

Lines 5492 to 5522 in 8e65a04

	@parameterized({
	'': (False,),
	'es6': (True,),
	})
	def test_webpack(self, es6):
	if es6:
	shutil.copytree(test_file('webpack_es6'), 'webpack')
	self.emcc_args += ['-sEXPORT_ES6', '-pthread', '-sPTHREAD_POOL_SIZE=1']
	outfile = 'src/hello.mjs'
	else:
	shutil.copytree(test_file('webpack'), 'webpack')
	outfile = 'src/hello.js'
	with common.chdir('webpack'):
	self.compile_btest('hello_world.c', ['-sEXIT_RUNTIME', '-sMODULARIZE', '-sENVIRONMENT=web,worker', '-o', outfile])
	self.run_process(shared.get_npm_cmd('webpack') + ['--mode=development', '--no-devtool'])
	shutil.copy('webpack/src/hello.wasm', 'webpack/dist/')
	self.run_browser('webpack/dist/index.html', '/report_result?exit:0')
	def test_vite(self):
	shutil.copytree(test_file('vite'), 'vite')
	with common.chdir('vite'):
	self.compile_btest('hello_world.c', ['-sEXPORT_ES6', '-sEXIT_RUNTIME', '-sMODULARIZE', '-o', 'hello.mjs'])
	self.run_process(shared.get_npm_cmd('vite') + ['build'])
	self.run_browser('vite/dist/index.html', '/report_result?exit:0')
	def test_rollup(self):
	shutil.copytree(test_file('rollup'), 'rollup')
	with common.chdir('rollup'):
	self.compile_btest('hello_world.c', ['-sEXPORT_ES6', '-sEXIT_RUNTIME', '-sMODULARIZE', '-o', 'hello.mjs'])
	self.run_process(shared.get_npm_cmd('rollup') + ['--config'])
	self.run_browser('rollup/index.html', '/report_result?exit:0')

. Perhaps we could extend those tests to include one that fails without this PR?

[lgarron]

I mean it will regress code size slightly. Each program we generate will get a few bytes bigger.

Ah, I see. From my impression, Emscripten already includes a lot of code to handle various environments and edge cases, and the number of bytes added would be rather mild.

(If anything, the presence of a node: prefix is an indication that the import and its direct call sites are effectively meant to be unreachable and can be elided for code compiled to a browser target (where code size matters much more). I'm not sure if any tools do this, though.)

Yes but is that try for any of the module that we use here in emscripten?

Certainly: https://www.npmjs.com/package/ws

That does sound like it can be quite bad. Can you point to bundler where this true, or an example with will fail with the current emscripten output? We have tests that run under rollup, vite and webpack:

I personally use esbuild, so here's an example using that:

// main.ts
import { fileURLToPath } from "url";
import { readFile } from "fs/promises";

const path = fileURLToPath(new URL("./package.json", import.meta.url));
console.log(JSON.parse(await readFile(path, "utf-8")));

package.json
{
	"type": "module",
	"dependencies": {
		"@types/node": "^22.13.1",
		"esbuild": "^0.25.0"
	}
}

Run:

npm install
npx esbuild --bundle --format=esm "main.ts"

This returns:

✘ [ERROR] Could not resolve "url"

    main.ts:1:30:
      1 │ import { fileURLToPath } from "url";
        ╵                               ~~~~~

  The package "url" wasn't found on the file system but is built into node. Are you trying to bundle
  for node? You can use "--platform=node" to do that, which will remove this error.

✘ [ERROR] Could not resolve "fs/promises"

    main.ts:2:25:
      2 │ import { readFile } from "fs/promises";
        ╵                          ~~~~~~~~~~~~~

  The package "fs/promises" wasn't found on the file system but is built into node. Are you trying
  to bundle for node? You can use "--platform=node" to do that, which will remove this error.

2 errors

I avoid --platform=node, for the reasons mentioned above (namely, that it can be ambiguous whether certain bare imports refer to node built-ins or packages), so I tend to use:

npx esbuild --bundle --format=esm "--external:node:*" "main.ts"

This works quite well (assuming, of course, that all node imports are prefixed with node:).

Perhaps we could extend those tests to include one that fails without this PR?

Happy to give it a go, what would it test? That the source code includes strings like "node:fs" but not "fs"? Or should it run code?

[sbc100]

I avoid --platform=node, for the reasons mentioned above (namely, that it can be ambiguous whether certain bare imports refer to node built-ins or packages), so I tend to use:

Just to confirm, you are trying to run the resulting bundled code on node? I assume so. Otherwise you can remove all the node imports (and code overhead) from the emscripten output using -sENVIRONMENT=web

[sbc100]

Ah, I see. From my impression, Emscripten already includes a lot of code to handle various environments and edge cases, and the number of bytes added would be rather mild.

You are correct the code size changes should minimal, and I'm tempted to agree with you say we should accept them in this case. But we have to very vigilant in emscripten because historically our code size has suffered from the death-by-a-thousand-cuts.

Certainly: https://www.npmjs.com/package/ws

Are you saying that node has a builtin ws package can could be confused for the npm-packaged one?

I avoid --platform=node, for the reasons mentioned above (namely, that it can be ambiguous whether certain bare imports refer to node built-ins or packages), so I tend to use:

So the errors only show up for you when you are targeting node but don't want to use the --platform=node flag designed for that purpose? We could go with that but it does seem a little contrived. Do you know of any other error case we could we could point to where bundlers get confused by the lack of node: prefix?

[lgarron]

Ah, I see. From my impression, Emscripten already includes a lot of code to handle various environments and edge cases, and the number of bytes added would be rather mild.

You are correct the code size changes should minimal, and I'm tempted to agree with you say we should accept them in this case. But we have to very vigilant in emscripten because historically our code size has suffered from the death-by-a-thousand-cuts.

I suppose that's fair! What's the best way to measure this for a PR?

Certainly: https://www.npmjs.com/package/ws

Are you saying that node has a builtin ws package can could be confused for the npm-packaged one?

Indeed!

I avoid --platform=node, for the reasons mentioned above (namely, that it can be ambiguous whether certain bare imports refer to node built-ins or packages), so I tend to use:

So the errors only show up for you when you are targeting node but don't want to use the --platform=node flag designed for that purpose? We could go with that but it does seem a little contrived.

Correct. I personally don't consider it contrived because its very much an everyday reality for me, since I generally publish libraries that work in node and in browsers using the same code. (And also write apps that can run code on both, when relevant.)

esbuild's --platform=node would often do the same thing as what I need, but there are times when I need esbuild not to avoid bundling a package that matches an unprefixed node built-in. So the documented semantics of --platform=node would produce incorrect code for my use case.

The --platform=node also comes with another effect already: it "disables the interpretation of the browser field in package.json", which may also affect runtime semantics. Sometimes this may actually a good thing, but it could also break bundled code. But it highlights that --platform=node is not semantically the thing as "avoid bundling node built-ins".

Do you know of any other error case we could we could point to where bundlers get confused by the lack of node: prefix?

I personally do not use bundlers other than esbuild, so I don't have any examples fresh off the press.

But hopefully I've made a clear case why bundlers cannot make a safe choice.

• If they always exclude packages that match node built-ins, then browser builds depending on packages with those names will be broken.
• If they assume all packages matching node built-ins should be bundled (or provide an option to do so), then this means some code will not be using node's built-ins when it expected to do so. At best either this fails immediately or the package code happens to be a close match (e.g. because it's a polyfill), but at worst this can introduce subtle bugs.
  • If the exclusion of built-ins is done by enumerating just packages in the code graph, then usage of a new built-in package becomes a breaking change for the bundling mechanism.
• If bundlers use heuristics, they can trigger false positives in either direction.
  • For example, suppose a bundler looks at whether a package is in package.json. Then adding a dependency may change the bundling of existing code. In addition, this wouldn't cover packages that are only in transitive dependencies.
  • Bundlers could instead look at lockfiles. I use bun and I think it's extremely unlikely that all (any?) bundlers read bun's lockfiles.
  • Bundlers could look at packages present in node_modules. However, it is quite possible for node_modules to include packages that are not in the current dependency tree (e.g. if you pulled or switched branches and haven't run npm install yet, or if you npm linked something).

In particular, note that a non-trivial codebase may include code (either in-tree or in a dependencies) that:

• Expects the node built-in in some places.
• Expects the package in some other places.

If such code is in dependencies, it can be very impractical to edit the call sites. (Vendoring can be expensive to maintain, sometimes requires bespoke package name mapping, and can lead to security issues more easily. Patching is supported by some tools, but I think not very common in the JS ecosystem — and can also be tricky to maintain.)

It's much more straightforward to aim for a world where:

• The node: prefix indicates node: built-ins.
• An unprefixed dependency refers to a package in the current JS registry (or import map) by default.

I understand this to be a current best practice. In fact, I would break CI in a lot of my projects if I tried to use node built-ins without a node: prefix. My linter errors on such an import by default: https://biomejs.dev/linter/rules/use-nodejs-import-protocol/

[sbc100]

Are you saying that node has a builtin ws package can could be confused for the npm-packaged one?

Indeed!

Perhaps I am misunderstand something here? I cannot seem to find any information on a ws package built into node and require('node:ws') doesn't work for me, even on node canary. What am I missing.

[sbc100]

What's the best way to measure this for a PR?

To measure the code size impact of a PR you first need to make sure you have the latest emsdk binaries installed (emsdk install tot) and then run ./tools/maint/rebaseline_tests.py. If can take a look at doing that for you if you not already setup to do emscripten development locally.

[sbc100]

One other blocker we might have this change it would require bumping the minimum version of node that we support. By default we target node v16 but users can opt into support for node versions going back as far as v10.19. See

emscripten/src/settings.js

Lines 1928 to 1934 in f3b1194

	// Specifies minimum node version to target for the generated code. This is
	// distinct from the minimum version required run the emscripten compiler.
	// This version aligns with the current Ubuuntu TLS 20.04 (Focal).
	// Version is encoded in MMmmVV, e.g. 181401 denotes Node 18.14.01.
	// Minimum supported value is 101900, which was released 2020-02-05 (see
	// feature_matrix.py).
	var MIN_NODE_VERSION = 160000;

It likely possible that we can bump this to v12, but we should perhaps open a separate issue for that.

[sbc100]

Looks like this would actually require node v16 to be sure it is supported: #23663 (comment)