module: exports check hasOwnProperty #31427

guybedford · 2020-01-20T19:02:34Z

This is a possible fix for the issues discussed in #31001 (comment), ensuring a strict property check.

I suspect the bug may be prototype pollution although I haven't been able to test this. I'm 80% sure this should resolve the issue though given the information.

I haven't included a test as this is more of a correctness feature... testing object pollution sounds more like something that could be covered by some kind of general JS pollution fuzz testing approach than something to be unit tested case-by-case.

I will be away for the next 8 hours, please feel free to merge if this works, or to revert the conditional exports unflagging PR if it doesn't.

//cc @Trott @richardlau @codebytere

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

richardlau

Tried this locally and it didn't fix the issue. I've run CITGM on bluebird with 13.6.0 (which doesn't contain #31001) and NODE_OPTIONS=--experimental-conditional-exports and it produces the same failure: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker-nobuild/727/

guybedford · 2020-01-20T19:20:07Z

@richardlau are you 100% sure this change didn’t fix the issue? The trySelf function should definitely be returning the early false return unless the package.json has an exports field. I really cannot see another way the execution paths would differ flagged versus unflagged. And yes adding the flag would replicate the issue as you have confirmed.

…

On Mon, Jan 20, 2020 at 21:12 Richard Lau ***@***.***> wrote: ***@***.**** requested changes on this pull request. Tried this locally and it didn't fix the issue. I've run CITGM on bluebird with 13.6.0 (which doesn't contain #31001 <#31001>) and NODE_OPTIONS=--experimental-conditional-exports and it produces the same failure: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker-nobuild/727/ — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#31427?email_source=notifications&email_token=AAESFSXG2VTZXOCYNQIW65TQ6XZQRA5CNFSM4KJIT5FKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCSL47FQ#pullrequestreview-345493398>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAESFSRI4UIKEXN4TNCEZ3LQ6XZQRANCNFSM4KJIT5FA> .

richardlau · 2020-01-20T19:37:03Z

are you 100% sure this change didn’t fix the issue?

Yes, I just double checked.

CITGM (bluebird only on our fastest platform (s390x)): https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/2231/

GeoffreyBooth · 2020-01-20T19:52:44Z

@richardlau Do you mind please providing specific steps to reproduce?

richardlau · 2020-01-20T20:02:39Z

Do you mind please providing specific steps to reproduce?

😅

I have git clones of nodejs/node and nodejs/citgm (with npm install already run inside the citgm clone).

Make the edits in the clone of nodejs/node (vi lib/internal/modules/cjs/loader.js)
build (make -j128 2>&1 | tee ../build.linux-x64.log)
install (tools/install.py install '' /home/riclau/sandbox/bin/testnode/) (this is so we can run with CITGM)
set path (export PATH=/home/riclau/sandbox/bin/testnode/bin:$PATH)
Change to the CITGM clone directory
run node bin/citgm.js bluebird

richardlau · 2020-01-20T20:13:08Z

It may be possible to reduce what bluebird is doing into a simpler testcase... From just looking at the code it appears that:

package.json contains "main": "./js/release/bluebird.js", and lists under devDependencies "bluebird": "^2.9.2",
tools/build.js's first line is var Promise = require("bluebird");

So I'm guessing that it used to "work" by falling back to the version of bluebird installed under node_modules (as one of the devDependencies) when Node.js couldn't load the non-existant main?

guybedford · 2020-01-20T20:50:24Z

Thanks @richardlau for the replication instructions there - I managed to finally track this down and have updated the PR with the fix.

Basically, for some reason, all package.json files are being detected as containing an undefined "exports" field in these cases. So shifting the check to be based on "exports" not being undefined instead of being a property fixes the issue.

Tracking down why "exports" is being set as an undefined property on the internally loaded package.json objects is likely a good idea - and entirely unrelated to this PR. I'm not sure if it's due to userland code or internal code.

Either way we are good to land with this fix if we can get this fast tracked now.

GeoffreyBooth · 2020-01-20T20:56:11Z

Assuming @richardlau can verify that this PR fixes the issue, I propose we fast-track.

richardlau · 2020-01-20T20:56:31Z

CITGM (bluebird only): https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/2232/

ljharb · 2020-01-20T21:01:59Z

lib/internal/modules/cjs/loader.js

@@ -435,7 +435,7 @@ function resolveBasePath(basePath, exts, isMain, trailingSlash, request) {

 function trySelf(parentPath, isMain, request) {
  const { data: pkg, path: basePath } = readPackageScope(parentPath) || {};
-  if (!pkg || 'exports' in pkg === false) return false;
+  if (!pkg || pkg.exports === undefined) return false;


this still would pass if an inherited exports property existed.

Suggested change

if (!pkg || pkg.exports === undefined) return false;

if (!pkg || !HasOwnProperty(pkg, 'exports')) return false;

(also bring in HasOwnProperty from primordials at the top)

Isn't that similar to the first commit (d3f5674) which doesn't work (pkg.exports does exist but has an undefined value)?

in that case you'd want both checks.

richardlau · 2020-01-20T21:12:16Z

Can confirm locally that this appears to fix the issue (and the results so far in https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/2232/ are in agreement).

For posterity I also came to a similar discovery about exports existing but being undefined (@guybedford PRed the second commit fix before I could post):

With the following hacky debug (on top of d3f5674):

diff --git a/lib/internal/modules/cjs/loader.js b/lib/internal/modules/cjs/loader.js
index 01d96a4..7c256e0 100644
--- a/lib/internal/modules/cjs/loader.js
+++ b/lib/internal/modules/cjs/loader.js
@@ -435,9 +435,9 @@ function resolveBasePath(basePath, exts, isMain, trailingSlash, request) {

 function trySelf(parentPath, isMain, request) {
   const { data: pkg, path: basePath } = readPackageScope(parentPath) || {};
-  if (!pkg || 'exports' in pkg === false) return false;
+  if (!pkg || !ObjectPrototypeHasOwnProperty(pkg, 'exports')) return false;
   if (typeof pkg.name !== 'string') return false;
-
+  if (pkg.name === 'bluebird') console.warn(pkg);
   let expansion;
   if (request === pkg.name) {
     expansion = '';

I got this out of the CITGM output:

warn:                         | {
warn:                         | name: 'bluebird',
warn:                         | main: './js/release/bluebird.js',
warn:                         | exports: undefined,
warn:                         | type: undefined
warn:                         | }

ljharb · 2020-01-20T21:19:33Z

What is supposed to happen if someone does Object.prototype.exports = {}?

BridgeAR · 2020-01-20T21:22:27Z

@guybedford the reason is that readPackageScope calls readPackage and that returns an object with four properties, no matter if they are defined in the package.json or not. One of those properties is exports.

codebytere · 2020-01-20T23:40:42Z

@ljharb PTAL - do you still think this needs both checks? i'm gonna land this in a bit otherwise :)

ljharb · 2020-01-21T00:09:24Z

@codebytere if the only reason for the bug was #31427 (comment) - and if readPackage only reads own properties - then both checks are not needed. However, if readPackage reads inherited properties, then i think both checks are still needed.

Trott · 2020-01-21T00:14:09Z

Would be good to see a test added for this.

Trott · 2020-01-21T00:53:13Z

Would be good to see a test added for this.

I took the liberty of adding a test. Hope that's OK. Feel free to remote it or modify it or whatever.

codebytere · 2020-01-21T01:00:09Z

@Trott do you not feel this should be backported to #31382 then? My other option was to take out the offending unflag commit but i was planning to pick this back if possible

Trott · 2020-01-21T01:02:46Z

@Trott do you not feel this should be backported to #31382 then? My other option was to take out the offending unflag commit but i was planning to pick this back if possible

I support putting this in the release rather than taking out the unflagging. But I'm fine with pulling out the unflagging and waiting until the next semver minor if you or anyone else is concerned. If it causes widespread breakage, the burden will fall to the releasers to get out a quick fix, so I'll defer to you on what the best thing to do here is. But I'm +1 on getting this into the release.

GeoffreyBooth · 2020-01-21T01:38:05Z

Unflagging conditional exports is very important for ESM adoption. Many package maintainers are waiting for conditional exports before they add ESM support to their packages, especially conditional exports in LTS; so I would err on the side of keeping the unflagging if it all possible.

richardlau · 2020-01-21T01:44:40Z

@Trott We can (should?) probably cut out a lot of stuff from the two package.json files in the test fixture (e.g. @petkaantonov 's email address). Maybe in a follow up if it would otherwise hold up landing this.

Trott · 2020-01-21T01:56:23Z

@Trott We can (should?) probably cut out a lot of stuff from the two package.json files in the test fixture (e.g. @petkaantonov 's email address). Maybe in a follow up if it would otherwise hold up landing this.

Agreed, that should happen (and probably rename it from something other than "bluebird") and also someone should follow up on @ljharb's observation and see if the other check is necessary or not.

nodejs-github-bot · 2020-01-21T05:00:15Z

CI: https://ci.nodejs.org/job/node-test-pull-request/28525/

nodejs-github-bot · 2020-01-21T05:10:11Z

CI: https://ci.nodejs.org/job/node-test-pull-request/28526/

Trott · 2020-01-21T05:31:22Z

https://ci.nodejs.org/job/node-test-pull-request/28525/ is green. Landing.

Trott · 2020-01-21T05:34:53Z

Landed in 343ddff...dcba128

Refs: nodejs#31001 (comment) PR-URL: nodejs#31427 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Shelley Vohr <codebytere@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

Refs: #31001 (comment) PR-URL: #31427 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Shelley Vohr <codebytere@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

guybedford · 2020-01-21T06:42:06Z

Thanks all for the help getting this one sorted.

Refs: #31001 (comment) PR-URL: #31427 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Shelley Vohr <codebytere@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

Refs: nodejs#31001 (comment) PR-URL: nodejs#31427 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Shelley Vohr <codebytere@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

Refs: #31001 (comment) PR-URL: #31427 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Shelley Vohr <codebytere@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

guybedford mentioned this pull request Jan 20, 2020

module: unflag conditional exports #31001

Closed

2 tasks

This comment has been minimized.

Sign in to view

richardlau requested changes Jan 20, 2020

View reviewed changes

GeoffreyBooth added the esm Issues and PRs related to the ECMAScript Modules implementation. label Jan 20, 2020

GeoffreyBooth added the fast-track PRs that do not need to wait for 48 hours to land. label Jan 20, 2020

This comment has been minimized.

Sign in to view

ljharb reviewed Jan 20, 2020

View reviewed changes

richardlau approved these changes Jan 20, 2020

View reviewed changes

richardlau mentioned this pull request Jan 20, 2020

v13.7.0 proposal #31382

Merged

This comment has been minimized.

Sign in to view

codebytere self-requested a review January 20, 2020 22:32

codebytere approved these changes Jan 20, 2020

View reviewed changes

cjihrig approved these changes Jan 20, 2020

View reviewed changes

codebytere requested a review from ljharb January 20, 2020 23:41

Trott approved these changes Jan 21, 2020 •

edited

Loading

View reviewed changes

This comment has been minimized.

Sign in to view

Trott closed this Jan 21, 2020

Trott force-pushed the exports-check-fix branch from 9cb334a to dcba128 Compare January 21, 2020 05:36

targos mentioned this pull request Jan 30, 2020

v12.15.0 release proposal #31368

Closed

MylesBorins mentioned this pull request Feb 8, 2020

v12.16.0 proposal #31691

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

module: exports check hasOwnProperty #31427

module: exports check hasOwnProperty #31427

guybedford commented Jan 20, 2020

This comment has been minimized.

richardlau left a comment

guybedford commented Jan 20, 2020 via email

richardlau commented Jan 20, 2020 •

edited

Loading

GeoffreyBooth commented Jan 20, 2020

richardlau commented Jan 20, 2020

richardlau commented Jan 20, 2020

guybedford commented Jan 20, 2020

GeoffreyBooth commented Jan 20, 2020

richardlau commented Jan 20, 2020 •

edited

Loading

This comment has been minimized.

ljharb Jan 20, 2020

richardlau Jan 20, 2020

ljharb Jan 20, 2020

richardlau commented Jan 20, 2020

ljharb commented Jan 20, 2020

BridgeAR commented Jan 20, 2020 •

edited

Loading

This comment has been minimized.

codebytere commented Jan 20, 2020

ljharb commented Jan 21, 2020

Trott commented Jan 21, 2020

Trott commented Jan 21, 2020

This comment has been minimized.

codebytere commented Jan 21, 2020

Trott commented Jan 21, 2020 •

edited

Loading

GeoffreyBooth commented Jan 21, 2020

richardlau commented Jan 21, 2020

Trott commented Jan 21, 2020 •

edited

Loading

This comment has been minimized.

nodejs-github-bot commented Jan 21, 2020

nodejs-github-bot commented Jan 21, 2020

Trott commented Jan 21, 2020

Trott commented Jan 21, 2020

guybedford commented Jan 21, 2020

	if (!pkg \|\| pkg.exports === undefined) return false;
	if (!pkg \|\| !HasOwnProperty(pkg, 'exports')) return false;

module: exports check hasOwnProperty #31427

module: exports check hasOwnProperty #31427

Conversation

guybedford commented Jan 20, 2020

Checklist

This comment has been minimized.

richardlau left a comment

Choose a reason for hiding this comment

guybedford commented Jan 20, 2020 via email

richardlau commented Jan 20, 2020 • edited Loading

GeoffreyBooth commented Jan 20, 2020

richardlau commented Jan 20, 2020

richardlau commented Jan 20, 2020

guybedford commented Jan 20, 2020

GeoffreyBooth commented Jan 20, 2020

richardlau commented Jan 20, 2020 • edited Loading

This comment has been minimized.

ljharb Jan 20, 2020

Choose a reason for hiding this comment

richardlau Jan 20, 2020

Choose a reason for hiding this comment

ljharb Jan 20, 2020

Choose a reason for hiding this comment

richardlau commented Jan 20, 2020

ljharb commented Jan 20, 2020

BridgeAR commented Jan 20, 2020 • edited Loading

This comment has been minimized.

codebytere commented Jan 20, 2020

ljharb commented Jan 21, 2020

Trott commented Jan 21, 2020

Trott commented Jan 21, 2020

This comment has been minimized.

codebytere commented Jan 21, 2020

Trott commented Jan 21, 2020 • edited Loading

GeoffreyBooth commented Jan 21, 2020

richardlau commented Jan 21, 2020

Trott commented Jan 21, 2020 • edited Loading

This comment has been minimized.

nodejs-github-bot commented Jan 21, 2020

nodejs-github-bot commented Jan 21, 2020

Trott commented Jan 21, 2020

Trott commented Jan 21, 2020

guybedford commented Jan 21, 2020

richardlau commented Jan 20, 2020 •

edited

Loading

richardlau commented Jan 20, 2020 •

edited

Loading

BridgeAR commented Jan 20, 2020 •

edited

Loading

Trott commented Jan 21, 2020 •

edited

Loading

Trott commented Jan 21, 2020 •

edited

Loading