Special treatment for package.json resolution and exports?

[ctavan]

📗 API Reference Docs Problem

• Version: 14.2.0
• Platform: any
• Subsystem: esm

Location

Section of the site where the content exists

Affected URL(s):

• https://nodejs.org/dist/latest-v14.x/docs/api/esm.html

Problem description

Concise explanation of what you found to be problematic

With the introduction of pkg.exports a module only exports the paths explicitly listed in pkg.exports, any other path can no longer be required. Let's have a look at an example:

Node 12.16.3:

> require.resolve('uuid/dist/v1.js');
'/example-project/node_modules/uuid/dist/v1.js'

Node 14.2.0:

> require.resolve('uuid/dist/v1.js');
Uncaught:
Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './dist/v1.js' is not defined by "exports" in /example-project/node_modules/uuid/package.json

So far, so good. The docs describe this behavior (although not super prominently):

Now only the defined subpath in "exports" can be imported by a consumer:
…
While other subpaths will error:
…

While this meets the expectations set out by the docs I stumbled upon package.json no longer being exported:

> require.resolve('uuid/package.json');
Uncaught:
Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './package.json' is not defined by "exports" in /example-project/node_modules/uuid/package.json

For whatever reason I wasn't assuming the documented rules to apply to package.json itself since I considered it package metadata, not package entrypoints whose visibility a package author would be able to control.

This new behavior creates a couple of issues with tools/bundlers that rely on meta information from package.json.

• So right now, packages that do contain information to be consumed externally (e.g. by bundlers) in their package.json have to add the package.json to the pkg.exports field.
• On the other hand bundlers/etc should then handle the case where a dependency doesn't export the package.json gracefully, since it might very well be the fact that a given package doesn't need to export any bundler meta information (and otherwise almost all packages on npm that could ever be used in a react-native project would have to add package.json to their exports).
• If bundlers however handle a missing package.json exports gracefully, I see the risk that many modules that currently rely on their package.json simply being externally consumable without additional effort might suddenly behave in odd ways when the meta information from package.json is no longer readable by bundlers (and no error is thrown).

Examples where this issue already surfaced:

• package.json is not defined by "exports" uuidjs/uuid#444
• Plugin incompatible with Node.js exports package.json field sveltejs/rollup-plugin-svelte#104
• Package subpath './package.json' is not defined by "exports" react-native-community/cli#1168
• Importing error in React Native starting from nanoevents 3.0 version ai/nanoevents#44 (comment)
• Warning: Ignored package due to "Package subpath './package.json' is not defined by "exports"" facebook/react-native#28710
• https://github.com/locize/i18next-locize-backend/pull/326/files
• Metro Bundler ignores make-plural eemeli/make-plural#15

Now the question is how to move forward with this?

1. One option would be to keep the current behavior and improve the documentation to explicitly warn about the fact, that package.json can no longer be resolved unless added to exports. EDIT: Already done in 1ffd182 / Node.js v14.3.0
2. Another option would be to consider adding an exception for package.json and always export it.

I had some discussion on slack with @ljharb and @wesleytodd but we didn't come to an ultimate conclusion yet 🤷‍♂️ .

---

• I would like to work on this issue and submit a pull request.

[ljharb]

cc @nodejs/modules-active-members

[wesleytodd]

other option would be to consider adding an exception for package.json and always export it.

To me this seems like a great solution. Having the package metadata is an awesome ergonomic feature of the current setup, and having module authors explicitly have to opt in would be a huge burden across the community. To me it seems like we would need a concrete reason not to have this exception. Can anyone think of a reason to make this case?

[bmeck]

@wesleytodd I think it just comes down to what is public/private still. People putting public data into a a package.json for things like tools to consume isn't really an issue. People putting configuration data like secrets is more the concern. I imagine it would still be able to be censored if people re-wrote the import still.

However, I'm unclear on the privacy model here since the benefit seems largely to be around bundlers which wouldn't run with the same constraints since they are ahead of time tools and general thought to access things in a more permissive manner than 2 independent modules with mutual distrust. It seems the problem is in part that these tools are using APIs that make them act as the same level of trust as any other module and other packages when they upgrade are removing permissions to view the package.json data (even if by accident). I think the concrete discussion here is if people should have to opt-out of package.json to avoid an accident prone workflow which is the inverse of all other resources in the package.

A different option since there is a specific use case that seems to need this is to have a flag of some kind for these ahead of time tools. Either to require.resolve which looks to be the cause of issues above, to node via CLI/ENV, or something else. I do think providing an exception would make things just work, but somewhat go against the privacy intent of "exports".

[hybrist]

My main concern with exposing it via exports is that there’s two options:

1. Make it non-configurable. ./package.json always has to map to ./package.json.
2. Tools that use require or import to load metadata may get custom files because a package decided to remap ./package.json.

The first option would force every package to treat its metadata file as a public API. Some users eslint config reuses what I put into my package.json#eslint section? Well, it’s exported so how can I blame them.

The second option means that tools may not actually get the metadata when they think they’re loading the metadata. It could be argued that it only affects “weird” packages but given a sufficiently large number of users that know about this “trick”, I can totally see people use it.

I think bundlers shouldn’t use require (“load code for this environment”) to load metadata. So I’d rather have a new API to load the package.json that belongs to a specifier/referrer combination, exposing logic we already have in the loader. That API could be used by bundlers etc to cleanly get the metadata without having to hijack require.

[ljharb]

I think the primary issue is that there's no way besides require.resolve to resolve the package.json for a package. Even path.join(require.resolve(pkg), 'package.json') won't work because some packages might have their "main" resolve to a subdirectory.

[guybedford]

Here's the recommended way to do this with ES modules:

import { readFileSync } from 'fs';
(async () => {
  const pkgPath = await import.meta.resolve('pkg/')
  console.log(pkgPath);
  console.log(readFileSync(new URL('package.json', pkgPath)).toString());
})();

The above also simplifies with TLA of course.

Currently the above only executes with --experimental-import-meta-resolve.

@nodejs/modules-active-members I think we should discuss unflagging this feature.

[ctavan]

I also want to clarify that the problems I have seen in the wild were always just about require.resolve('pkg/package.json') in order to then load that file from the filesystem. I didn't see anybody trying to directly require('pkg/package.json') to really load the json data as a module.

[ljharb]

@guybedford import.meta.resolve('pkg/') would fail if the package didn't have a main/dot, wouldn't it? or, if the ./ was mapped to a different directory, like ./src?

[guybedford]

@ljharb no, the trailing / is specially specified to allow resolving package boundaries.

[ljharb]

@guybedford require.resolve('es-get-iterator/') in latest node throws Package exports for '$PWD/node_modules/es-get-iterator' do not define a './' subpath. I would expect import.meta.resolve to behave identically, so the same capabilities exist in both CJS and ESM.

iow, import.meta.resolve only solves for ESM, not CJS, so it's not a solution to this problem.