Resolving multiple package.json "main" fields

[jamiebuilds]

TL;DR: A new compiler option mainFields for selecting multiple fields in package.json instead of just package.json#main.

---

There are lots of related issues to this one (which I link to below), but I want to focus on just this specific proposal.

---

Packages often look like this:

{
  "name": "my-package",
  "main": "dist/cjs/index.js",
  "module": "dist/esm/index.js"
  "source": "src/index.ts"
}

Notice how we have multiple fields which specify multiple entry points. These entry points all refer to the same code, just in different compile states and configurations.

Many tools use these fields in order to find the entry point that they care about. For example, tools like Webpack and Rollup will use package.json#module in order to find ES modules. Other tools will use fields like package.json#source (or src) for local package development.

While these fields aren't part of the official Node module resolution algorithm. They are a community convention which has proven to be useful in lots of scenarios.

For TypeScript, one such scenario that this would be useful for is with multi-package repos or "monorepos". These are repositories where the code for multiple npm packages exist and are symlinked together locally.

/project/
  package.json
  /packages/
    /package-one/
      package.json
      /node_modules/
        /package-two/ -> ../../package-two (symlink)
    /package-two/
      package.json

Inside each package, you'll generally have a src/ directory that gets compiled to dist/

/package-two/
  package.json
  /src/
    index.ts
  /dist/
    index.js
    index.d.ts

Right now it is really painful to use TypeScript with one of these repos. This is because TypeScript will use the package.json#main to resolve to the packages dist folders. The problem with this is that the dist folders might not exist and if they do exist they might not be compiled from the most recent version of src.

To work around this today you can add a index.ts file in the root of each of your packages to point to the right location and make sure that the root index.ts file does not get shipped to npm.

/package-two/
  index.ts
  /src/index.ts

// package-two/index.ts
export * from './src/index'

It sucks that you need this file, and if you ever forget to create it in a new package, you'll revert back to really crap behavior.

If, instead of all that, TypeScript supported a new compiler option mainFields which looked like:

{
  "compilerOptions": {
    "mainFields": ["source", "main"]
  }
}

Note: Webpack has this same configuration option

You could add package.json#source (in addition to package.json#main) and resolve it to the right location locally.

The algorithm would look like this:

For each mainField:

1. Check if the package.json has a field with that name
2. If the package.json does not have the field, continue to next mainField
3. If it field exists, check for a file at that location.
4. If no file at that location exists, continue to the next mainField
5. If the file exists, use that file as the resolved module and stop looking

I think this is the relevant code:

https://github.com/Microsoft/TypeScript/blob/b363f4f9cd6ef98f9451ccdcc7321d151195200b/src/compiler/moduleNameResolver.ts#L987-L1014

Related Issues:

• Path mapping in yarn workspaces #21137 "Path mapping in yarn workspaces"
• Module resolution for sub-packages picks d.ts file when .ts file is available #20248 "Module resolution for sub-packages picks d.ts file when .ts file is available"
• Support .mjs output #18442 "Support .mjs output"

[DanielRosenwasser]

I think the monorepo scenario you have in mind is covered by #3469, but I think there's still room to discuss a package.json resolution strategy.

The problem is that for every dependency that relies on that, the end consumer needs to cover each mainField. It's not a huge problem, but it's the same sort of mental overhead of figuring out that you need to install @types/node for your dependencies to type-check correctly.

[jamiebuilds]

That's not going to work for many monorepos unless you can deal with circular dependencies somehow. It'd also mean that we'd have to duplicate the work of adding dependencies in our package.json and the tsconfig.json files. I'm also not sure how it'd work with our multiple build targets

[aluanhaddad]

For providing intellisense, when no declarations are available, wouldn't it always be better to resolve to the source, if available, than to the compiled output and would that be true irrespective of the source being written in TypeScript?

Even with --allowJs the compiled output is usually resolved when there are no type declarations. This does not only affect developers working with monorepos, it affects anyone consuming packages without declarations.

Of course, it depends on the package, but when using say, "Go to Definition", it is very common to be taken to a UMD bundle and that is probably the least desirable result.

Having said that, there are too many of these damn fields!

Here is a totally non-exhaustive list of main fields that should be considered applicable

"main"
"browser"
"types"
"module"
"jsnext:main"
"es2015"
"unpkg"
"typings"

[lski]

I have been hitting this issue in my Typescript/Webpack/Babel/React app (yes its a mix lol). Using the resolve.mainFields setting in webpack worked great with tree-shaking when writing a none-typescript app, as more and more libraries on NPM (including each of my own) are moving towards supporting the different entry points. I then moved back to Typescript I realised I lost all the benefits as I was stuck importing the full compiled library.

I know the whole main/browser/module has been in flux for a while, especially with jsnext:main/es2015, but it seems to be solidifying around main/browser/module, with things such as unpkg being used for something very specific.

Just my two pence obviously, to show there is a desire for this recommendation. This took 2 days of research just to find out I couldnt do the type of tree-shaking I can do relatively out of the box with webpack and JavaScript, meaning the mental energy used was actually working out the different behaviour in Typescript, rather than knowing about main fields.

[mhegazy]

We have recently added support for building sourceMaps for declaration files, see #22658. We have also added support for tools to go through these declaration files and land on original sources. The net result here is you open a project, hit F12 on an imported declaration, and land in the source code for the referenced module.
We are also working on a rationalized system of project-to-project references in #3469.

There are two main implications of loading of loading .ts files from source instead of .d.ts files, namely 1. configuration has to be the same between the referencing project and the referenced project, since you are basically including the code from one into the other, and 2. the sizes of the state that an IDE (tsserver) needs to keep in memory for these projects can be large, and merging them together increases that and does not give the tools a way to split some of that and unload it for instance.

With the proposed solutions in #22658 and #3469, .d.ts files are still the main interface between projects, allowing for separate compilations and separate configurations. it also allows the tools to build logical boundaries between projects, and can independently jettison some of their state as needed to manage resource consumption.

That said, this whole effort is just starting, we need to support other language service operations like find-all-references and rename on the mapped .d.ts files, we also need to find a way to keep these .d.ts files updated when the .ts files are updated to give the ideal experience.

[jamiebuilds]

.d.ts files won't ever be able to solve this use case because they require building before usage, when you have circular graphs that need to be run sequentially, that is impossible

[cellvia]

please add support for mainFields ... im trying to have vscode resolve to module src files instead of main, because i need to get proper intellisense from external modules jsdoc annotations. i cant use main / dist because that is precompiled by babel which mangles everything. please please 🙏 this doesnt seem like it should be too hard... just need the option to alter the property to look up in package.json for resolving !

[typeofweb]

Has there been any progress on this issue?

[gluons]

Oh! module field resolving sound great! I really want separate CommonJS and ES module to each single file.

For now, I'm hacking. 😅

export default MyExport;
module.exports = exports.default;

[cellvia]

im also curious the progress on this

[thysultan]

Another alternative is for TypeScript to support source-maps and use the jsdoc annotation from the related source-locations. As has been mentioned before – often times an isomorphic libraries pkg.main field points to a bundled file that is a minified soup of comment-less code.

[DavidWells]

I am running into a similar issue.

I have a package that has browser & node compatible versions in the dist directory.

https://github.com/DavidWells/analytics/blob/master/packages/analytics-core/package.json#L20-L25

When I run my build, a version of the package is built for the browser (referencing dom/window etc) and a version is build for node.js

The different versions (node vs browser) have different slightly different types.

I have a "types" key set in the package.json file pointing to a x.d.ts definition file but VS code only seems to pickup the on that matches the main key in package.json

How can I get all files in my dist directory using the proper d.ts files?

[RIP21]

I hit the following issue which I think is related to this issue/proposal

Quote from my Stackoverflow question
https://stackoverflow.com/questions/61491159/how-to-stop-typescript-compiler-from-reporting-compilation-errors-in-symlinked-m

Question:
I have a monorepo controlled by rush.js with PNPM as a package manager.

I used to have all shared modules to be precompiled into cjs, esm, dts targets. But this approach has some flaws, so I decided to keep them as untouched sources, and set their main entry in package.json to be "main": "./src/index.ts|x".
At the same time, I used react-app-rewired to tell Webpack to compile only those symlinked libraries from node_modules using babel and everything works perfectly. Jest is happy too.

The problem that I've got tho, is that when I run tsc for some reason compiler goes deep into the symlinked local packages and reports A LOT of issues (even tho they are compiling without any issues if you run their tsc).

TSForkWebpackPlugin reported similar issues for create-react-app but I ignored them using reportFiles config option using react-app-rewired and thought it was some sort of bug on plugin site, but it seems it's not.

I added all sorts of glob patterns to exclude like **/node_modules/@namespace/** and node_modules/@namespace/** and node_modules/@namespace none of those worked.
"skipLibCheck": true is there too.

My tsconfig.json for reference

{
  "compilerOptions": {
    "incremental": true,
    "baseUrl": "src",
    "downlevelIteration": true,
    "lib": ["esnext", "dom", "dom.iterable"],
    "module": "esnext",
    "target": "esnext",
    "sourceMap": true,
    "allowJs": true,
    "esModuleInterop": true,
    "isolatedModules": true,
    "jsx": "preserve",
    "moduleResolution": "node",
    "forceConsistentCasingInFileNames": false,
    "noImplicitReturns": true,
    "noImplicitThis": true,
    "noImplicitAny": true,
    "noUnusedParameters": true,
    "noUnusedLocals": true,
    "strictNullChecks": true,
    "suppressImplicitAnyIndexErrors": true,
    "skipLibCheck": true,
    "noEmit": true,
    "preserveSymlinks": true,
    "resolveJsonModule": true,
    "allowSyntheticDefaultImports": true,
    "strict": true
  },
  "exclude": [
    "node_modules"
  ],
  "include": ["src"]
}

Soultion:

The only solution to this issue is to emit d.ts files and add types entry to package.json so TSC thinks that these packages are compiled libraries and no complaints since. It's not the worst workaround, although we're losing around 25-30 seconds on CI.

Maybe there is a way to filter files from being reported? Like TSWebpackForkPlugin does? E.g. reportFiles glob patterns to make these workarounds easier if it's really hard to fix in a right way?

[pie6k]

The solution that worked for me is keeping both uncompiled and compiled files in the same folder (lib) instead of having both src and dist.

TLDR: During development - all .js files are removed. During publishing - all .ts files are ignored.

Here is the flow:

For simplicity, let's consider our package has only one, index.ts source file.

I keep it in <ROOT>/lib/index.ts or <ROOT>/packages/foo/lib/index.ts.

Than in package.json I pass "main": "./lib". Note I'm not passing exact file name. TS will be able to resolve to index.ts in development. Bundlers will be able to resolve to index.js in published package.

During development - I have a script that clears all .js files in lib (therefore you cannot use .js files in development).

This makes typescript properly point to ts file.

When releasing the package - I run build which will add .js files next to their .ts counterparts.

In .npmignore - I ignore all lib/**.ts files, so in final, published version there are only .js files in lib.

In 'production' - package.json main field which points to ./lib will properly resolve to ./lib/index.js.

In .gitignore - I ignore all lib/**.js files - so in my git repo I don't have .js files in lib published if I forget to run my clean script before pushing.

[speigg]

Ideally, resolution configuration via both mainFields and custom user conditions are supported, so that Conditional Exports can be leveraged.

// package.json
{
  "exports": {
    "typings": {
          "source": "./index.ts",
          "default": "./index.d.ts"
     },
    "import": "./index.mjs",
    "require": "./index.cjs",
  }
}

// tsconfig.json
{
  "compilerOptions": {
    "resolveCustomConditions": ["source"],
    "resolveMainFields":  ["typings","module","main"]
  }
}

[evelant]

I'm running into this issue. I've got a react-native app that I'd like to build with tsc (I want to try ts-plus) but some react-native pacakges rely on the non-standard "react-native": field in package.json to take precendent for resolving their main field. When I build with tsc I end up getting failures and crashes due to browser specific code since tsc will only resolve "main":.

Obviously it would be better if react-native would not use non-standard fields for resolution but unfortunately that's just the way it is.

[adonespitogo]

Any solution to this? I'm also building a mono repo and want to be able to use intellisense without having to rebuild all of my sub repos each time.

[cefn]

I shared some other suggestions for what this package.json convention might be in #51750 which I'm closing in favour of this long-lived issue.

[cefn]

@jamiebuilds doesn't the use of an index.ts at the top level of a package get defeated if you have a "types" declaration. The stated resolution order of Typescript suggests the "types" field has priority over index.ts.

Also what problems have you encountered, if any, from distributing the index.ts file in the top level of the package? Does it cause Typescript to break when it depends on the package (e.g. since it resolves the actual source code but that source might not be compatible with the downstream Typescript compiler config?

[cefn]

I asked the question here if anyone can contribute... https://stackoverflow.com/questions/74688869/resolving-packages-directly-to-typescript-source-in-monorepo-package-json-files

[yuvron]

Have you managed to find a solution?
@cefn

[cefn]

I've been optimistic about https://www.typescriptlang.org/tsconfig#customConditions but that came too late for my own repository.

I ended up 'leaning in' to the pattern that everything has to be built at all times, and adopting tooling to help with this. It's fully crazy complexity that I'd rather not have to manage and it's pointless (given all the source is Typescript) but there we are.

Since restructuring the project to use --build, there may be some 'implicit' local resolution happening I guess.

[jpikl]

I had similar issue and was able to solve it using yarn (modern version) as a package manager.

Here is my package.json

{
  "main": "src/index",
  "publishConfig": {
    "main": "dist/index",
    "types": "dist/index"
  }
}

During local development, TS will us the main field.

When the package is published using yarn npm publish, Yarn will automatically swap the publishConfig, so the result looks like

{
  "main": "dist/index",
  "types": "dist/index"
}

You will need to install yarn typescript plugin for the type field to work.

[cefn]

In pnpm this is also supported, but is ignored by npm :(

[jo0ger]

Is there a conclusion to this question?

[cefn]

My own conclusion was

• use pnpm, which supports publishConfig swapping out the main src/index.ts to point to dist/index.js only when it's ready for distribution
• use tricks to put a file where typescript will detect it first and prioritise it during module resolution (like the original post or like my approach)
• use custom conditions, allowing a local compilation condition to target src/index.ts, while normal conditions would resolve to the dist/index.js file

I speculate that the original feature request (to have a unique resolution direct to typescript native files internally to monorepos) is directly fulfilled by the support for custom conditions. @jamiebuilds is that fair?