Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

esm: add chaining to loaders #42623

Merged

Conversation

JakobJingleheimer
Copy link
Contributor

@JakobJingleheimer JakobJingleheimer commented Apr 6, 2022

Notable changes

  • ESMLoader now provides more detailed error messages for hooks, including the specific file and hook from which the error was caused (ex the hook broke the chain; but not SyntaxError within the hook—which was already working). In order to facilitate this:
    • ESMLoader's internal collections of hooks have changed from hook fn[]Array<{ url: specifier, fn: hook }>
    • ESMLoader::import()'s return has changed from ModuleNamespace[] | ModuleNamespaceArray<{ url: specifier, fn: hook }> | ModuleNamespace

TODOs:

  • hook up next()s
  • custom loaders FIFO → LIFO
  • detect & throw on unsignalled short-circuit
  • update existing hook output error messages to include newly available specifier
  • update code docs
  • update Error docs
  • update ESM docs
  • verify/fix node-land breakages caused the change to ESMLoader::import() (I think this will not be a breaking change to user-land as the functionality seen in user-land continues to be where ESMLoader::import() returns a single ModuleNamespace)
  • update current tests
  • add new tests for chaining

Trying it out

CLI
./node \
    --loader=/PATH/TO/loaderA.mjs \
    --loader=/PATH/TO/loaderB.mjs \
    --input-type=module \
    -e "import fs from 'node:fs'; console.log(2)"
loaderA.mjs
export const resolve = async function resolveA(
	specifier,
	context,
	next,
) {
	console.log('resolveA running');
	return {
		shortCircuit: true,
		url: 'file:///foo/bar.js',
	};
}
export const load = async function loadA(
	resolvedUrl,
	context,
	next,
) {
	console.log('loadA running');
	return {
		format: 'module',
		shortCircuit: true,
		source: 'export default 42',
	}
}
loaderB.mjs
export const resolve = async function resolveB(
	specifier,
	context,
	next,
) {
	console.log('resolveB running');
	return next(specifier);
}
export const load = async function loadB(
	resolvedUrl,
	context,
	next,
) {
	console.log('loadB running');
	return next(resolvedUrl);
}

42 should be printed to stdout when the hooks are properly chained; when disabled, the node:fs module should be printed to stdout.

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Apr 6, 2022

Review requested:

  • @nodejs/modules
  • @nodejs/loaders

@nodejs-github-bot nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Apr 6, 2022
@JakobJingleheimer JakobJingleheimer added esm Issues and PRs related to the ECMAScript Modules implementation. loaders Issues and PRs related to ES module loaders labels Apr 6, 2022
src/node_options.h Outdated Show resolved Hide resolved
lib/internal/errors.js Outdated Show resolved Hide resolved
lib/internal/modules/esm/loader.js Outdated Show resolved Hide resolved
@JakobJingleheimer JakobJingleheimer force-pushed the esm/add-chaining-to-loaders branch 2 times, most recently from ff0c53e to e2c1337 Compare April 6, 2022 18:26
@JakobJingleheimer
Copy link
Contributor Author

Hrrrm. I've got all the existing es-module tests updated except for es-module/test-esm-symlink-type.js: all of its cases are failing with the same TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension "" for […]. That error seems legit based on where it's coming from (ESMLoader doesn't support extensionless files), but the test seems to think it should be okay.

I didn't see anything related to symlinks when making my changes, so I'm not sure what's up 🤔

@bmeck
Copy link
Member

bmeck commented Apr 6, 2022 via email

@JakobJingleheimer
Copy link
Contributor Author

I presume so based on your question. Do you know whereabouts I could check to confirm that is or isn't happening?

By the time it gets to ESMLoader::load()defaultLoad()getFormat() it (supposedly—I didn't print it) doesn't have an extension.

@JakobJingleheimer JakobJingleheimer added the notable-change PRs with changes that should be highlighted in changelogs. label Apr 8, 2022
@JakobJingleheimer JakobJingleheimer marked this pull request as ready for review April 10, 2022 11:09
@JakobJingleheimer

This comment was marked as outdated.

@targos targos marked this pull request as draft April 10, 2022 11:43
@targos targos marked this pull request as ready for review April 10, 2022 11:43
@targos

This comment was marked as resolved.

@JakobJingleheimer

This comment was marked as outdated.

Copy link
Member

@GeoffreyBooth GeoffreyBooth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven’t reviewed the tests, but I’ve written more than enough notes so far 😄 Overall this is quite good; I agree with the general thrust and design. It shouldn’t take too much work to clean everything up and get this merged.

doc/api/errors.md Outdated Show resolved Hide resolved
doc/api/esm.md Outdated Show resolved Hide resolved
doc/api/esm.md Outdated Show resolved Hide resolved
doc/api/esm.md Outdated Show resolved Hide resolved
doc/api/esm.md Outdated Show resolved Hide resolved
lib/internal/modules/esm/loader.js Outdated Show resolved Hide resolved
lib/internal/modules/esm/loader.js Outdated Show resolved Hide resolved
lib/internal/modules/run_main.js Outdated Show resolved Hide resolved
lib/internal/process/esm_loader.js Outdated Show resolved Hide resolved
lib/internal/util/types.js Outdated Show resolved Hide resolved
@JakobJingleheimer
Copy link
Contributor Author

If possible, yes. I think #37468 made it into v16? (if so, it should be okay)

@targos
Copy link
Member

targos commented Jul 19, 2022

Yes it did. I cherry-picked this change 👍🏻

targos pushed a commit that referenced this pull request Jul 19, 2022
PR-URL: #42623
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
@cspotcode
Copy link

Double-checking, is #43763 going to land in v16 as well? Otherwise it'll break projects.

@targos
Copy link
Member

targos commented Jul 20, 2022

It wasn't going to be because by default commits are supposed to bake in a Current release before we consider them for LTS, but I just added the lts-watch label and will make sure it lands in v16 too.

targos pushed a commit that referenced this pull request Jul 31, 2022
PR-URL: #42623
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
targos added a commit that referenced this pull request Aug 2, 2022
Notable changes:

crypto:
  * (SEMVER-MINOR) remove Node.js-specific webcrypto extensions (Filip Skokan) #43310
  * (SEMVER-MINOR) add CFRG curves to Web Crypto API (Filip Skokan) #42507
  * (SEMVER-MINOR) make authTagLength optional for CC20P1305 (Tobias Nießen) #42427
  * (SEMVER-MINOR) align webcrypto RSA key import/export with other implementations (Filip Skokan) #42816
deps:
  * update undici to 5.4.0 (Node.js GitHub Bot) #43262
  * update undici to 5.3.0 (Node.js GitHub Bot) #43197
dns:
  * (SEMVER-MINOR) export error code constants from `dns/promises` (Feng Yu) #43176
doc:
  * add F3n67u to collaborators (Feng Yu) #43953
  * deprecate coercion to integer in process.exit (Daeyeon Jeong) #43738
  * (SEMVER-MINOR) deprecate diagnostics_channel object subscribe method (Stephen Belanger) #42714
  * add LiviaMedeiros to collaborators (LiviaMedeiros) #43039
  * add @kuriyosh to collaborators (Yoshiki Kurihara) #42824
  * add RafaelGSS to collaborators (RafaelGSS) #42718
  * add @meixg to collaborators (Xuguang Mei) #42576
errors:
  * (SEMVER-MINOR) add support for cause in aborterror (James M Snell) #41008
esm:
  * (SEMVER-MINOR) add chaining to loaders (Jacob Smith) #42623
events:
  * (SEMVER-MINOR) expose CustomEvent on global with CLI flag (Daeyeon Jeong) #43885
  * (SEMVER-MINOR) add `CustomEvent` (Daeyeon Jeong) #43514
  * (SEMVER-MINOR) propagate abortsignal reason in new AbortError ctor in events (James M Snell) #41008
fs:
  * (SEMVER-MINOR) propagate abortsignal reason in new AbortSignal constructors (James M Snell) #41008
  * (SEMVER-MINOR) make params in writing methods optional (LiviaMedeiros) #42601
  * (SEMVER-MINOR) add `read(buffer[, options])` versions (LiviaMedeiros) #42768
http:
  * (SEMVER-MINOR) add drop request event for http server (theanarkh) #43806
  * (SEMVER-MINOR) add diagnostics channel for http client (theanarkh) #43580
  * (SEMVER-MINOR) add perf_hooks detail for http request and client (theanarkh) #43361
  * (SEMVER-MINOR) add uniqueHeaders option to request and createServer (Paolo Insogna) #41397
http2:
  * (SEMVER-MINOR) propagate abortsignal reason in new AbortError constructor (James M Snell) #41008
  * (SEMVER-MINOR) compat support for array headers (OneNail) #42901
lib:
  * (SEMVER-MINOR) propagate abortsignal reason in new AbortError constructor in blob (James M Snell) #41008
  * (SEMVER-MINOR) add abortSignal.throwIfAborted() (James M Snell) #40951
  * (SEMVER-MINOR) improved diagnostics_channel subscribe/unsubscribe (Stephen Belanger) #42714
module:
  * (SEMVER-MINOR) add isBuiltIn method (hemanth.hm) #43396
module,repl:
  * (SEMVER-MINOR) support 'node:'-only core modules (Colin Ihrig) #42325
net:
  * (SEMVER-MINOR) add drop event for net server (theanarkh) #43582
  * (SEMVER-MINOR) add ability to reset a tcp socket (pupilTong) #43112
node-api:
  * (SEMVER-MINOR) emit uncaught-exception on unhandled tsfn callbacks (Chengzhong Wu) #36510
perf_hooks:
  * (SEMVER-MINOR) add PerformanceResourceTiming (RafaelGSS) #42725
report:
  * (SEMVER-MINOR) add more heap infos in process report (theanarkh) #43116
src:
  * (SEMVER-MINOR) add --openssl-legacy-provider option (Daniel Bevenius) #40478
  * (SEMVER-MINOR) define fs.constants.S_IWUSR & S_IRUSR for Win (Liviu Ionescu) #42757
src,doc,test:
  * (SEMVER-MAJOR) add --openssl-shared-config option (Daniel Bevenius) #43124
stream:
  * (SEMVER-MINOR) use cause options in AbortError constructors (James M Snell) #41008
  * (SEMVER-MINOR) add iterator helper find (Nitzan Uziely) #41849
  * (SEMVER-MINOR) add writableAborted (Robert Nagy) #40802
test:
  * (SEMVER-MINOR) add initial test module (Colin Ihrig) #42325
test_runner:
  * (SEMVER-MINOR) expose `describe` and `it` (Moshe Atlow) #43420
  * (SEMVER-MINOR) add initial CLI runner (Colin Ihrig) #42658
  * (SEMVER-MINOR) support 'only' tests (Colin Ihrig) #42514
timers:
  * (SEMVER-MINOR) propagate signal.reason in awaitable timers (James M Snell) #41008
util:
  * (SEMVER-MINOR) add tokens to parseArgs (John Gee) #43459
  * (SEMVER-MINOR) add parseArgs module (Benjamin Coe) #42675
v8:
  * (SEMVER-MINOR) add v8.startupSnapshot utils (Joyee Cheung) #43329
  * (SEMVER-MINOR) export more fields in getHeapStatistics (theanarkh) #42784
worker:
  * (SEMVER-MINOR) add hasRef() to MessagePort (Darshan Sen) #42849

PR-URL: TODO
targos added a commit that referenced this pull request Aug 6, 2022
Notable changes:

Adds `util.parseArgs` helper for higher level command-line argument
parsing.

Contributed by Benjamin Coe, John Gee, Darcy Clarke, Joe Sepi,
Kevin Gibbons, Aaron Casanova, Jessica Nahulan, and Jordan Harband.
#42675

Node.js ESM Loader hooks now support multiple custom loaders, and
composition is achieved via "chaining": `foo-loader` calls `bar-loader`
calls `qux-loader` (a custom loader _must_ now signal a short circuit
when intentionally not calling the next). See the ESM docs
(https://nodejs.org/dist/latest-v16.x/docs/api/esm.html) for details.

Contributed by Jacob Smith, Geoffrey Booth, and Bradley Farias.
#42623

The `node:test` module, which was initially introduced in Node.js
v18.0.0, is now available with all the changes done to it up to Node.js
v18.7.0.

To better align Node.js' experimental implementation of the Web Crypto
API with other runtimes, several changes were made:

* Support for CFRG curves was added, with the `'Ed25519'`, `'Ed448'`,
  `'X25519'`, and `'X448'` algorithms.
* The proprietary `'NODE-DSA'`, `'NODE-DH'`, `'NODE-SCRYPT'`,
  `'NODE-ED25519'`, `'NODE-ED448'`, `'NODE-X25519'`, and `'NODE-X448'`
  algorithms were removed.
* The proprietary `'node.keyObject'` import/export format was removed.

Contributed by Filip Skokan.
#42507
#43310

Updated Corepack to 0.12.1 - #43965
Updated ICU to 71.1 - #42655
Updated npm to 8.15.0 - #43917
Updated Undici to 5.8.0 - #43886

(SEMVER-MINOR) crypto: make authTagLength optional for CC20P1305 (Tobias Nießen) [#42427](#42427)
(SEMVER-MINOR) crypto: align webcrypto RSA key import/export with other implementations (Filip Skokan) [#42816](#42816)
(SEMVER-MINOR) dns: export error code constants from `dns/promises` (Feng Yu) [#43176](#43176)
doc: deprecate ercion to integer in process.exit (Daeyeon Jeong) [#43738](#43738)
(SEMVER-MINOR) doc: deprecate diagnostics\_channel object subscribe method (Stephen Belanger) [#42714](#42714)
(SEMVER-MINOR) errors: add support for cause in aborterror (James M Snell) [#41008](#41008)
(SEMVER-MINOR) events: expose CustomEvent on global with CLI flag (Daeyeon Jeong) [#43885](#43885)
(SEMVER-MINOR) events: add `CustomEvent` (Daeyeon Jeong) [#43514](#43514)
(SEMVER-MINOR) events: propagate abortsignal reason in new AbortError ctor in events (James M Snell) [#41008](#41008)
(SEMVER-MINOR) fs: propagate abortsignal reason in new AbortSignal constructors (James M Snell) [#41008](#41008)
(SEMVER-MINOR) fs: make params in writing methods optional (LiviaMedeiros) [#42601](#42601)
(SEMVER-MINOR) fs: add `read(buffer[, options])` versions (LiviaMedeiros) [#42768](#42768)
(SEMVER-MINOR) http: add drop request event for http server (theanarkh) [#43806](#43806)
(SEMVER-MINOR) http: add diagnostics channel for http client (theanarkh) [#43580](#43580)
(SEMVER-MINOR) http: add perf\_hooks detail for http request and client (theanarkh) [#43361](#43361)
(SEMVER-MINOR) http: add uniqueHeaders option to request and createServer (Paolo Insogna) [#41397](#41397)
(SEMVER-MINOR) http2: propagate abortsignal reason in new AbortError constructor (James M Snell) [#41008](#41008)
(SEMVER-MINOR) http2: compat support for array headers (OneNail) [#42901](#42901)
(SEMVER-MINOR) lib: propagate abortsignal reason in new AbortError constructor in blob (James M Snell) [#41008](#41008)
(SEMVER-MINOR) lib: add abortSignal.throwIfAborted() (James M Snell) [#40951](#40951)
(SEMVER-MINOR) lib: improved diagnostics\_channel subscribe/unsubscribe (Stephen Belanger) [#42714](#42714)
(SEMVER-MINOR) module: add isBuiltIn method (hemanth.hm) [#43396](#43396)
(SEMVER-MINOR) moduleepl**: support 'node:'-only core modules (Colin Ihrig) [#42325](#42325)
(SEMVER-MINOR) net: add drop event for net server (theanarkh) [#43582](#43582)
(SEMVER-MINOR) net: add ability to reset a tcp socket (pupilTong) [#43112](#43112)
(SEMVER-MINOR) nodepi**: emit uncaught-exception on unhandled tsfn callbacks (Chengzhong Wu) [#36510](#36510)
(SEMVER-MINOR) perfhooks**: add PerformanceResourceTiming (RafaelGSS) [#42725](#42725)
(SEMVER-MINOR) report: add more heap infos in process report (theanarkh) [#43116](#43116)
(SEMVER-MINOR) src: add --openssl-legacy-provider option (Daniel Bevenius) [#40478](#40478)
(SEMVER-MINOR) src: define fs.constants.S\_IWUSR & S\_IRUSR for Win (Liviu Ionescu) [#42757](#42757)
(SEMVER-MINOR) srcoc,test**: add --openssl-shared-config option (Daniel Bevenius) [#43124](#43124)
(SEMVER-MINOR) stream: use cause options in AbortError constructors (James M Snell) [#41008](#41008)
(SEMVER-MINOR) stream: add iterator helper find (Nitzan Uziely) [#41849](#41849)
(SEMVER-MINOR) stream: add writableAborted (Robert Nagy) [#40802](#40802)
(SEMVER-MINOR) timers: propagate signal.reason in awaitable timers (James M Snell) [#41008](#41008)
(SEMVER-MINOR) v8: add v8.startupSnapshot utils (Joyee Cheung) [#43329](#43329)
(SEMVER-MINOR) v8: export more fields in getHeapStatistics (theanarkh) [#42784](#42784)
(SEMVER-MINOR) worker: add hasRef() to MessagePort (Darshan Sen) [#42849](#42849)

PR-URL: #44098
targos added a commit that referenced this pull request Aug 6, 2022
Notable changes:

Adds `util.parseArgs` helper for higher level command-line argument
parsing.

Contributed by Benjamin Coe, John Gee, Darcy Clarke, Joe Sepi,
Kevin Gibbons, Aaron Casanova, Jessica Nahulan, and Jordan Harband.
#42675

Node.js ESM Loader hooks now support multiple custom loaders, and
composition is achieved via "chaining": `foo-loader` calls `bar-loader`
calls `qux-loader` (a custom loader _must_ now signal a short circuit
when intentionally not calling the next). See the ESM docs
(https://nodejs.org/dist/latest-v16.x/docs/api/esm.html) for details.

Contributed by Jacob Smith, Geoffrey Booth, and Bradley Farias.
#42623

The `node:test` module, which was initially introduced in Node.js
v18.0.0, is now available with all the changes done to it up to Node.js
v18.7.0.

To better align Node.js' experimental implementation of the Web Crypto
API with other runtimes, several changes were made:

* Support for CFRG curves was added, with the `'Ed25519'`, `'Ed448'`,
  `'X25519'`, and `'X448'` algorithms.
* The proprietary `'NODE-DSA'`, `'NODE-DH'`, `'NODE-SCRYPT'`,
  `'NODE-ED25519'`, `'NODE-ED448'`, `'NODE-X25519'`, and `'NODE-X448'`
  algorithms were removed.
* The proprietary `'node.keyObject'` import/export format was removed.

Contributed by Filip Skokan.
#42507
#43310

Updated Corepack to 0.12.1 - #43965
Updated ICU to 71.1 - #42655
Updated npm to 8.15.0 - #43917
Updated Undici to 5.8.0 - #43886

(SEMVER-MINOR) crypto: make authTagLength optional for CC20P1305 (Tobias Nießen) [#42427](#42427)
(SEMVER-MINOR) crypto: align webcrypto RSA key import/export with other implementations (Filip Skokan) [#42816](#42816)
(SEMVER-MINOR) dns: export error code constants from `dns/promises` (Feng Yu) [#43176](#43176)
doc: deprecate ercion to integer in process.exit (Daeyeon Jeong) [#43738](#43738)
(SEMVER-MINOR) doc: deprecate diagnostics\_channel object subscribe method (Stephen Belanger) [#42714](#42714)
(SEMVER-MINOR) errors: add support for cause in aborterror (James M Snell) [#41008](#41008)
(SEMVER-MINOR) events: expose CustomEvent on global with CLI flag (Daeyeon Jeong) [#43885](#43885)
(SEMVER-MINOR) events: add `CustomEvent` (Daeyeon Jeong) [#43514](#43514)
(SEMVER-MINOR) events: propagate abortsignal reason in new AbortError ctor in events (James M Snell) [#41008](#41008)
(SEMVER-MINOR) fs: propagate abortsignal reason in new AbortSignal constructors (James M Snell) [#41008](#41008)
(SEMVER-MINOR) fs: make params in writing methods optional (LiviaMedeiros) [#42601](#42601)
(SEMVER-MINOR) fs: add `read(buffer[, options])` versions (LiviaMedeiros) [#42768](#42768)
(SEMVER-MINOR) http: add drop request event for http server (theanarkh) [#43806](#43806)
(SEMVER-MINOR) http: add diagnostics channel for http client (theanarkh) [#43580](#43580)
(SEMVER-MINOR) http: add perf\_hooks detail for http request and client (theanarkh) [#43361](#43361)
(SEMVER-MINOR) http: add uniqueHeaders option to request and createServer (Paolo Insogna) [#41397](#41397)
(SEMVER-MINOR) http2: propagate abortsignal reason in new AbortError constructor (James M Snell) [#41008](#41008)
(SEMVER-MINOR) http2: compat support for array headers (OneNail) [#42901](#42901)
(SEMVER-MINOR) lib: propagate abortsignal reason in new AbortError constructor in blob (James M Snell) [#41008](#41008)
(SEMVER-MINOR) lib: add abortSignal.throwIfAborted() (James M Snell) [#40951](#40951)
(SEMVER-MINOR) lib: improved diagnostics\_channel subscribe/unsubscribe (Stephen Belanger) [#42714](#42714)
(SEMVER-MINOR) module: add isBuiltIn method (hemanth.hm) [#43396](#43396)
(SEMVER-MINOR) moduleepl**: support 'node:'-only core modules (Colin Ihrig) [#42325](#42325)
(SEMVER-MINOR) net: add drop event for net server (theanarkh) [#43582](#43582)
(SEMVER-MINOR) net: add ability to reset a tcp socket (pupilTong) [#43112](#43112)
(SEMVER-MINOR) nodepi**: emit uncaught-exception on unhandled tsfn callbacks (Chengzhong Wu) [#36510](#36510)
(SEMVER-MINOR) perfhooks**: add PerformanceResourceTiming (RafaelGSS) [#42725](#42725)
(SEMVER-MINOR) report: add more heap infos in process report (theanarkh) [#43116](#43116)
(SEMVER-MINOR) src: add --openssl-legacy-provider option (Daniel Bevenius) [#40478](#40478)
(SEMVER-MINOR) src: define fs.constants.S\_IWUSR & S\_IRUSR for Win (Liviu Ionescu) [#42757](#42757)
(SEMVER-MINOR) srcoc,test**: add --openssl-shared-config option (Daniel Bevenius) [#43124](#43124)
(SEMVER-MINOR) stream: use cause options in AbortError constructors (James M Snell) [#41008](#41008)
(SEMVER-MINOR) stream: add iterator helper find (Nitzan Uziely) [#41849](#41849)
(SEMVER-MINOR) stream: add writableAborted (Robert Nagy) [#40802](#40802)
(SEMVER-MINOR) timers: propagate signal.reason in awaitable timers (James M Snell) [#41008](#41008)
(SEMVER-MINOR) v8: add v8.startupSnapshot utils (Joyee Cheung) [#43329](#43329)
(SEMVER-MINOR) v8: export more fields in getHeapStatistics (theanarkh) [#42784](#42784)
(SEMVER-MINOR) worker: add hasRef() to MessagePort (Darshan Sen) [#42849](#42849)

PR-URL: #44098
targos added a commit that referenced this pull request Aug 15, 2022
Notable changes:

Adds `util.parseArgs` helper for higher level command-line argument
parsing.
Contributed by Benjamin Coe, John Gee, Darcy Clarke, Joe Sepi,
Kevin Gibbons, Aaron Casanova, Jessica Nahulan, and Jordan Harband.
#42675

Node.js ESM Loader hooks now support multiple custom loaders, and
composition is achieved via "chaining": `foo-loader` calls `bar-loader`
calls `qux-loader` (a custom loader _must_ now signal a short circuit
when intentionally not calling the next). See the ESM docs
(https://nodejs.org/dist/latest-v16.x/docs/api/esm.html) for details.
Contributed by Jacob Smith, Geoffrey Booth, and Bradley Farias.
#42623

The `node:test` module, which was initially introduced in Node.js
v18.0.0, is now available with all the changes done to it up to Node.js
v18.7.0.

To better align Node.js' experimental implementation of the Web Crypto
API with other runtimes, several changes were made:
* Support for CFRG curves was added, with the `'Ed25519'`, `'Ed448'`,
  `'X25519'`, and `'X448'` algorithms.
* The proprietary `'NODE-DSA'`, `'NODE-DH'`, `'NODE-SCRYPT'`,
  `'NODE-ED25519'`, `'NODE-ED448'`, `'NODE-X25519'`, and `'NODE-X448'`
  algorithms were removed.
* The proprietary `'node.keyObject'` import/export format was removed.
Contributed by Filip Skokan.
#42507
#43310

Updated Corepack to 0.12.1 - #43965
Updated ICU to 71.1 - #42655
Updated npm to 8.15.0 - #43917
Updated Undici to 5.8.0 - #43886

(SEMVER-MINOR) crypto: make authTagLength optional for CC20P1305 (Tobias Nießen) #42427
(SEMVER-MINOR) crypto: align webcrypto RSA key import/export with other implementations (Filip Skokan) #42816
(SEMVER-MINOR) dns: export error code constants from `dns/promises` (Feng Yu) #43176
doc: deprecate coercion to integer in process.exit (Daeyeon Jeong) #43738
(SEMVER-MINOR) doc: deprecate diagnostics_channel object subscribe method (Stephen Belanger) #42714
(SEMVER-MINOR) errors: add support for cause in aborterror (James M Snell) #41008
(SEMVER-MINOR) events: expose CustomEvent on global with CLI flag (Daeyeon Jeong) #43885
(SEMVER-MINOR) events: add `CustomEvent` (Daeyeon Jeong) #43514
(SEMVER-MINOR) events: propagate abortsignal reason in new AbortError ctor in events (James M Snell) #41008
(SEMVER-MINOR) fs: propagate abortsignal reason in new AbortSignal constructors (James M Snell) #41008
(SEMVER-MINOR) fs: make params in writing methods optional (LiviaMedeiros) #42601
(SEMVER-MINOR) fs: add `read(buffer[, options])` versions (LiviaMedeiros) #42768
(SEMVER-MINOR) http: add drop request event for http server (theanarkh) #43806
(SEMVER-MINOR) http: add diagnostics channel for http client (theanarkh) #43580
(SEMVER-MINOR) http: add perf_hooks detail for http request and client (theanarkh) #43361
(SEMVER-MINOR) http: add uniqueHeaders option to request and createServer (Paolo Insogna) #41397
(SEMVER-MINOR) http2: propagate abortsignal reason in new AbortError constructor (James M Snell) #41008
(SEMVER-MINOR) http2: compat support for array headers (OneNail) #42901
(SEMVER-MINOR) lib: propagate abortsignal reason in new AbortError constructor in blob (James M Snell) #41008
(SEMVER-MINOR) lib: add abortSignal.throwIfAborted() (James M Snell) #40951
(SEMVER-MINOR) lib: improved diagnostics_channel subscribe/unsubscribe (Stephen Belanger) #42714
(SEMVER-MINOR) module: add isBuiltIn method (hemanth.hm) #43396
(SEMVER-MINOR) module,repl: support 'node:'-only core modules (Colin Ihrig) #42325
(SEMVER-MINOR) net: add drop event for net server (theanarkh) #43582
(SEMVER-MINOR) net: add ability to reset a tcp socket (pupilTong) #43112
(SEMVER-MINOR) node-api: emit uncaught-exception on unhandled tsfn callbacks (Chengzhong Wu) #36510
(SEMVER-MINOR) perf_hooks: add PerformanceResourceTiming (RafaelGSS) #42725
(SEMVER-MINOR) report: add more heap infos in process report (theanarkh) #43116
(SEMVER-MINOR) src: add --openssl-legacy-provider option (Daniel Bevenius) #40478
(SEMVER-MINOR) src: define fs.constants.S_IWUSR & S_IRUSR for Win (Liviu Ionescu) #42757
(SEMVER-MINOR) src,doc,test: add --openssl-shared-config option (Daniel Bevenius) #43124
(SEMVER-MINOR) stream: use cause options in AbortError constructors (James M Snell) #41008
(SEMVER-MINOR) stream: add iterator helper find (Nitzan Uziely) #41849
(SEMVER-MINOR) stream: add writableAborted (Robert Nagy) #40802
(SEMVER-MINOR) timers: propagate signal.reason in awaitable timers (James M Snell) #41008
(SEMVER-MINOR) v8: add v8.startupSnapshot utils (Joyee Cheung) #43329
(SEMVER-MINOR) v8: export more fields in getHeapStatistics (theanarkh) #42784
(SEMVER-MINOR) worker: add hasRef() to MessagePort (Darshan Sen) #42849

PR-URL: #44098
targos added a commit that referenced this pull request Aug 16, 2022
Notable changes:

Adds `util.parseArgs` helper for higher level command-line argument
parsing.
Contributed by Benjamin Coe, John Gee, Darcy Clarke, Joe Sepi,
Kevin Gibbons, Aaron Casanova, Jessica Nahulan, and Jordan Harband.
#42675

Node.js ESM Loader hooks now support multiple custom loaders, and
composition is achieved via "chaining": `foo-loader` calls `bar-loader`
calls `qux-loader` (a custom loader _must_ now signal a short circuit
when intentionally not calling the next). See the ESM docs
(https://nodejs.org/dist/latest-v16.x/docs/api/esm.html) for details.
Contributed by Jacob Smith, Geoffrey Booth, and Bradley Farias.
#42623

The `node:test` module, which was initially introduced in Node.js
v18.0.0, is now available with all the changes done to it up to Node.js
v18.7.0.

To better align Node.js' experimental implementation of the Web Crypto
API with other runtimes, several changes were made:
* Support for CFRG curves was added, with the `'Ed25519'`, `'Ed448'`,
  `'X25519'`, and `'X448'` algorithms.
* The proprietary `'NODE-DSA'`, `'NODE-DH'`, `'NODE-SCRYPT'`,
  `'NODE-ED25519'`, `'NODE-ED448'`, `'NODE-X25519'`, and `'NODE-X448'`
  algorithms were removed.
* The proprietary `'node.keyObject'` import/export format was removed.
Contributed by Filip Skokan.
#42507
#43310

Updated Corepack to 0.12.1 - #43965
Updated ICU to 71.1 - #42655
Updated npm to 8.15.0 - #43917
Updated Undici to 5.8.0 - #43886

(SEMVER-MINOR) crypto: make authTagLength optional for CC20P1305 (Tobias Nießen) #42427
(SEMVER-MINOR) crypto: align webcrypto RSA key import/export with other implementations (Filip Skokan) #42816
(SEMVER-MINOR) dns: export error code constants from `dns/promises` (Feng Yu) #43176
doc: deprecate coercion to integer in process.exit (Daeyeon Jeong) #43738
(SEMVER-MINOR) doc: deprecate diagnostics_channel object subscribe method (Stephen Belanger) #42714
(SEMVER-MINOR) errors: add support for cause in aborterror (James M Snell) #41008
(SEMVER-MINOR) events: expose CustomEvent on global with CLI flag (Daeyeon Jeong) #43885
(SEMVER-MINOR) events: add `CustomEvent` (Daeyeon Jeong) #43514
(SEMVER-MINOR) events: propagate abortsignal reason in new AbortError ctor in events (James M Snell) #41008
(SEMVER-MINOR) fs: propagate abortsignal reason in new AbortSignal constructors (James M Snell) #41008
(SEMVER-MINOR) fs: make params in writing methods optional (LiviaMedeiros) #42601
(SEMVER-MINOR) fs: add `read(buffer[, options])` versions (LiviaMedeiros) #42768
(SEMVER-MINOR) http: add drop request event for http server (theanarkh) #43806
(SEMVER-MINOR) http: add diagnostics channel for http client (theanarkh) #43580
(SEMVER-MINOR) http: add perf_hooks detail for http request and client (theanarkh) #43361
(SEMVER-MINOR) http: add uniqueHeaders option to request and createServer (Paolo Insogna) #41397
(SEMVER-MINOR) http2: propagate abortsignal reason in new AbortError constructor (James M Snell) #41008
(SEMVER-MINOR) http2: compat support for array headers (OneNail) #42901
(SEMVER-MINOR) lib: propagate abortsignal reason in new AbortError constructor in blob (James M Snell) #41008
(SEMVER-MINOR) lib: add abortSignal.throwIfAborted() (James M Snell) #40951
(SEMVER-MINOR) lib: improved diagnostics_channel subscribe/unsubscribe (Stephen Belanger) #42714
(SEMVER-MINOR) module: add isBuiltIn method (hemanth.hm) #43396
(SEMVER-MINOR) module,repl: support 'node:'-only core modules (Colin Ihrig) #42325
(SEMVER-MINOR) net: add drop event for net server (theanarkh) #43582
(SEMVER-MINOR) net: add ability to reset a tcp socket (pupilTong) #43112
(SEMVER-MINOR) node-api: emit uncaught-exception on unhandled tsfn callbacks (Chengzhong Wu) #36510
(SEMVER-MINOR) perf_hooks: add PerformanceResourceTiming (RafaelGSS) #42725
(SEMVER-MINOR) report: add more heap infos in process report (theanarkh) #43116
(SEMVER-MINOR) src: add --openssl-legacy-provider option (Daniel Bevenius) #40478
(SEMVER-MINOR) src: define fs.constants.S_IWUSR & S_IRUSR for Win (Liviu Ionescu) #42757
(SEMVER-MINOR) src,doc,test: add --openssl-shared-config option (Daniel Bevenius) #43124
(SEMVER-MINOR) stream: use cause options in AbortError constructors (James M Snell) #41008
(SEMVER-MINOR) stream: add iterator helper find (Nitzan Uziely) #41849
(SEMVER-MINOR) stream: add writableAborted (Robert Nagy) #40802
(SEMVER-MINOR) timers: propagate signal.reason in awaitable timers (James M Snell) #41008
(SEMVER-MINOR) v8: add v8.startupSnapshot utils (Joyee Cheung) #43329
(SEMVER-MINOR) v8: export more fields in getHeapStatistics (theanarkh) #42784
(SEMVER-MINOR) worker: add hasRef() to MessagePort (Darshan Sen) #42849

PR-URL: #44098
@localjo
Copy link

localjo commented Aug 23, 2022

It seems that this PR had breaking changes, but got merged into the 16.17.0 release. See the discussion in yarnpkg/berry#4778 for details.

@JakobJingleheimer
Copy link
Contributor Author

@localjo it did indeed have breaking changes. It's experimental, so it doesn't require a major version bump. The author of yarn was aware of the incoming change (he is on this team).

@localjo
Copy link

localjo commented Aug 23, 2022

@JakobJingleheimer It seems that the breaking changes are breaking a lot of people's builds though. See the discussion here: yarnpkg/berry#4778

Whether it should have been merged or not, I don't know. But I'm mentioning it here in case people are looking for info about why their projects stopped building, and how to solve it. In my case, I solved my issue by locking Node in my CI environment to 16.16.0. Less than ideal, but it works for me.

@GeoffreyBooth
Copy link
Member

Whether it should have been merged or not, I don't know.

It was an intentional breaking change. Loaders are still an experimental API.

@localjo
Copy link

localjo commented Aug 23, 2022

Fair enough. It seems a handful of packages must be using that experimental API and the people (including myself) who are importing those packages are unaware. Likely either an RTFM problem on my part, or lack of documentation on the part of the publishers of the packages. Anyway, thanks for the clarification.

@wbt
Copy link
Contributor

wbt commented Sep 2, 2022

Breaking changes, especially intentional ones, should not be merged without a breaking-change semver update. This also broke my build process, with GitHub Actions automatically incorporating the latest version of Node. Code that previously worked with no indications about it being experimental was suddenly both labeled as experimental and broken, without a corresponding change in the codebase.

@JakobJingleheimer
Copy link
Contributor Author

JakobJingleheimer commented Sep 2, 2022

@wbt you are mistaken. This feature has been marked experimental since its inception multiple years ago. This change was also highlighted very prominently at the very top of the release notes as breaking and specifically how and how to update. Experimental features are not semver major.

If your build process is sensitive to experimental changes, you should pin your version appropriately. For example, we do not release breaking experimental changes in semver patch, only semver minor (and if it so happens, major).

If you are dependent on some intermediary that is leveraging an experimental feature and did not pin or release appropriately, take it up with that dependency.

EDIT: If you would like to be notified of notable and breaking changes in Loaders/Customisation API, please subscribe to this issue nodejs/loaders#103

@nodejs nodejs locked and limited conversation to collaborators Sep 2, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. esm Issues and PRs related to the ECMAScript Modules implementation. lib / src Issues and PRs related to general changes in the lib or src directory. loaders Issues and PRs related to ES module loaders needs-ci PRs that need a full CI run. notable-change PRs with changes that should be highlighted in changelogs. semver-minor PRs that contain new features and should be released in the next minor version.
Projects
None yet
Development

Successfully merging this pull request may close these issues.