NodeNext resolution failed to resolve dual-package correctly

[Jack-Works]

Bug Report

NodeNext resolution failed to resolve dual-package correctly

🔎 Search Terms

NodeNext

🕗 Version & Regression Information

• Never worked

💻 Code

https://github.com/Jack-Works/ts-nodenext-wrong-resolution-reproduction

where node_modules/testpkg/package.json is

{
    "name": "testpkg",
    "exports": {
      ".": {
        "types": "./dist/type.d.ts",
        "require": "./dist/common.cjs",
        "import": "./dist/module.mjs"
      }
    }
  }

TypeScript should resolve type.d.ts in dual mode instead of CommonJS synthetic export.

🙁 Actual behavior

src/index.ts:2:1 - error TS2349: This expression is not callable.
  Type 'typeof import("testpkg/dist/type")' has no call signatures.

🙂 Expected behavior

No error

[DanielRosenwasser]

Okay, I think I'm starting to see why @weswigham is so insistent on import/require having separate entries for types as well.

I believe that what's going wrong here has to do with how the package interprets a .ts file, and that's based on the package's type field - which is unlisted in testpkg.

As a result, a .d.ts file will be interpreted as a CJS module, and the implicit esModuleInterop behavior kicks in where the entire module comes in as a default import.

If you set testpkg to have the field "type": "module", then it'll work in this case. If you need to support CJS better, you may need to also create a separate declaration file for each implementation file.

"exports": {
  ".": {
    "import": {
        "types": "./dist/module-type.d.mts",
        "default": "./dist/module.mjs"
    },
    "require": {
        "types": "./dist/commonjs-type.d.cts"
        "default": "./dist/common.cjs",
    }
  }
}

Similar discussion over here.

The docs may need to be updated in light of this.

[Jack-Works]

This is not just a documentation problem.

In the case I provided (which I believe is very common in the ecosystem), ESM and CommonJS have the same export list under two different import modes.
Therefore, TypeScript should be able to interpret it in both ways, meaning the following examples should both work:

// test.cjs
const f = require('testpkg')
f() // f is callable.

// test.mjs
import f from 'testpkg'
f() // f is also callable.

[weswigham]

@Jack-Works your export map is essentially telling TS to load both your esm and cjs entry point as cjs, since it's only providing a cjs format types file. You must have a separate TS file (with matching format) for each, as @DanielRosenwasser says, if you want accurate checking.

[olivier-martin-sf]

I am one of the maintainers of a "dual-package" and after reading this thread I have some questions related to the thread comments:

1. Is that correct that we should have one type declaration per file?
2. Should the type declarations uses the matching import/export semantics for the module system they are matching? (export for .d.mjs and module.exports for .d.cjs)
3. Does TypeScript will assume that a file ending with .d.ts is going to be an ESM if the type property is set to module in the package manifest?

More detailed below:

The package in question is declared as ES Module (the type property is set to module in the package manifest file). The package doesn't have main entry points per se and is structured as follows:

├── dist/
    ├── foo/
        ├── someFile.cjs  # Common JS
        ├── someFile.d.ts # Type declaration  
        ├── someFile.js   # ESM

In this case, both the CJS and the ESM implementation expose the exact same API (actually the CJS implementation is transpiled from the ESM one via Rollup inside a compiler).

That's how the exports look like for each implementation:

# someFile.js (ESM)
export default class C {}

# someFile.cjs (CJS)
class C {}
module.exports = C;

That's how the type declaration looks like:

export default class C {}

And that's how we configured the exports map in the package manifest:

"./*": {
    "types": "./dist/*.d.ts",
    "import": "./dist/*.js",
    "require": "./dist/*.cjs"
}

As soon as TS 4.8 has been released I tried to switch moduleResolution to NodeNext but it seems like I am hitting issues due to the fact that I am only using one declaration file if my understanding is correct. So basically, the format of my export map is wrong and I should opt for something close to what @DanielRosenwasser suggested.

I am also asking as the type declaration is being generated from our compiler and I will have to think about how we can generate one for CJS if we have to.

For instance, while trying to use the package in a project that uses CommonJS I am getting the following error from tsserver:

The current file is a CommonJS module whose imports will produce 'require' calls; however, the referenced file is an ECMAScript module and cannot be imported with 'require' (ts1479)

Thanks in advance for your input.

[Jack-Works]

Does TypeScript will assume that a file ending with .d.ts is going to be an ESM if the type property is set to module in the package manifest?

Yes.

"./*": {
    "import": {
        "types": "./dist/*.d.ts",
//                           ~~
        "default": "./dist/*.js"
    },
    "require": {
        "types": "./dist/*.d.cts",
//                           ~~~
        "default": "./dist/*.cjs"
    }
}

This will fix the problem. You should emit double declaration files (CJS once, and ESM once) with different extension names to make things work (which I think is very incomprehensible).

If all of your imports can be imported in both CJS and ESM mode and set correctly under NodeNext resolution, I think you can just copy the file and rename it to .d.cts to make things work.

I tried to set-up a dual package correctly (run tsc twice to emit both cjs and esm, also make sure NodeNext resolution check runs on both cjs and esm mode)

https://github.com/Jack-Works/ts-dual-package-example but it's very crazy.

[olivier-martin-sf]

@weswigham @andrewbranch do you confirm that's what we should be doing? Just want to make sure I am getting this right.

On a side note, do you mind giving pointers in which part of the TS codebase that resolution process is implemented?

[weswigham]

You must have a separate TS file (with matching format) for each [entrypoint] ... if you want accurate checking.

All module resolution is in moduleNameResolver.ts.

[olivier-martin-sf]

Thx for your answers @weswigham and @Jack-Works. I think my confusion come from the latest code snippet from this page of the handbook: https://www.typescriptlang.org/docs/handbook/esm-node.html#packagejson-exports-imports-and-self-referencing

By default, TypeScript overlays the same rules with import conditions - if you write an import from an ES module, it will look up the import field, and from a CommonJS module, it will look at the require field. If it finds them, it will look for a colocated declaration file. If you need to point to a different location for your type declarations, you can add a "types" import condition

// package.json
{
    "name": "my-package",
    "type": "module",
    "exports": {
        ".": {
            // Entry-point for TypeScript resolution - must occur first!
            "types": "./types/index.d.ts",
            // Entry-point for `import "my-package"` in ESM
            "import": "./esm/index.js",
            // Entry-point for `require("my-package") in CJS
            "require": "./commonjs/index.cjs",
        },
    },
    // CJS fall-back for older versions of Node.js
    "main": "./commonjs/index.cjs",
    // Fall-back for older versions of TypeScript
    "types": "./types/index.d.ts"
}

Which seems to contradict what is being discussed here. Should the documentation be updated?

[weswigham]

Yes.

[olivier-martin-sf]

Thanks for clarifying this @weswigham!

I have a last confusion to clear up regarding around the .d.cts semantics.
First, will it work to copy .d.ts type declaration to .d.cts without changing the imports / exports semantics?

Basically I am wondering if that type declaration:

// someFile.d.ts
import {Base as _Base } from '@scope/runtime';
import A from 'package/paths/a';

export default class Foo extends _Base {
  getFoo(): A
}

Could be ported to .d.cts just by renaming it, or if I should go with something like:

// someFile.d.cts
import {Base as _Base } from '@scope/runtime';
import A from 'package/paths/a';

declare class Foo extends _Base {
  getFoo(): A
}

exports = Foo;

[andrewbranch]

The correctness of any declaration is determined by the implementation file it types, so you can’t answer that question without looking at the .cjs file.

[olivier-martin-sf]

Thanks, @andrewbranch, it makes sense. I guess my question is more related to the syntax one should be used for exports/imports in .d.cts if it differs from .d.ts, are there any docs or examples around this I can refer to?

[andrewbranch]

I’m still not sure the question totally makes sense—to ask whether any declaration file should use this or that syntax is just to ask what syntax exists in its implementation file counterpart. There’s never a case where a declaration file is designed from a blank slate and then an implementation follows. In the simplest cases the declaration file and the implementation file will both be generated simultaneously from a TS source file by tsc. The only time we need to think about writing a declaration file by hand is when typing an external or native dependency, as contributors to DefinitelyTyped do. But in those cases, we have a reference implementation or API to match, so the question of what should go in the declaration file is objectively answerable by looking at the implementation.

In your case Rollup should be handling all of this for you, right? Are you trying to figure out if Rollup is doing the right thing?

[olivier-martin-sf]

Our use case is a bit specific so let me try to clarify. My team works on a source-to-source compiler that's generating JavaScript (in both CJS & ESM) and a type declaration (.d.ts) from a single JSON file. So for every matching JSON file, it generates the following files:

• file.js: ESM JS implementation for runtime
• file.cjs: CommonJS JS implementation for runtime
• file.d.ts: type declaration

The file.cjs is transpiled from the file.js using Rollup. Both files expose the same API and export a single class.

That's the structure of the generated JS for the ESM module format:

// file.js
import {Base as _Base } from '@scope/runtime';
import A from 'package/paths/a';

export default class Foo extends _Base {
   /**
    * @returns {A}
    */
  getFoo() {
  // impl...
  return new A();
  }
}

That's the structure of the generated JS for the CommonJS module format:

// file.cjs
var core = require('@scope/runtime');
var _A = require('package/paths/a');

// Interop fn added by Rollup
function _interopDefaultLegacy (e) { return e && typeof e === 'object' && 'default' in e ? e : { 'default': e }; }

var _A__default = /*#__PURE__*/_interopDefaultLegacy(_A);

class Foo extends core.Base{
   /**
    * @returns {A}
    */
  getFoo() {
  // impl...
  return new _A__default['default']();
  }
}

module.exports = Foo;

That's what the generated type declaration looks like:

// file.d.ts
import {Base as _Base } from '@scope/runtime';
import A from 'package/paths/a';

export default class Foo extends _Base {
  getFoo(): A
}

We don't rely on tsc for the generation of those files. In other words, the type declaration file is emitted by our compiler and not by TSC. Our compiler starts by generating both the ESM module and its associated type declaration from the parsed AST. Then it generates the CommonJS JS implementation by transpiling from the ESM.

We are using that compiler to generate a bunch of files packaged in an npm package. I updated the exports map to use what's being recommended in the package and it works indeed. Now I am trying to figure out how to generate those .d.cts file with the correct semantic.

I've tried running tsc to only emit declaration over the .cjs file but it's leading to some ts errors as tsc doesn't have enough context and is being to narrow. I wrote a babel plugin that would change the exports so that:

// file.d.ts
export default class C{}

Becomes:

// file.d.cts
declare class C{}
export = C;

And I am trying to see what I should do about the imports:

import {Base as _Base } from '@scope/runtime';
import A from 'package/paths/a';

And eventually, port that plugin to Rollup and back it up in our compiler transpilation stage.
Hope that helps provide more context. Thanks a lot for all your help on this.

[andrewbranch]

Because the .cjs file says module.exports = Foo, export = Foo would be correct for the .d.cts file, as it sounds like you’ve figured out already. (The reason I couldn’t tell you that without seeing the .cjs file is you could have very well chosen another ESM→CJS strategy where default exports become module.exports.default, not module.exports.) Default exports are the thing that is messiest to try to share, so 100% you cannot get by with only one declaration file to describe your ESM and CJS file simultaneously.

[olivier-martin-sf]

Thx a lot @andrewbranch, I really appreciate your explanations on this. My question was related to the import in the .d.cts file? Can I leave the ones from the .d.ts untouched or is there a need to transform them so that TypeScript will resolve them correctly?

[andrewbranch]

You can’t generally assume that an import in an ESM file will resolve the same when you convert that file to a CJS context. This is by design, of course; you’re taking advantage of conditional exports yourself. If you leave the imports untouched, TypeScript will resolve them “correctly,” but will it resolve to the same thing as it did in an ESM context? If not, will the import syntax used be appropriate for the different module it resolves to? Will the resulting file type check and do what you expect? You have to answer these questions yourself for every individual import.

[olivier-martin-sf]

That makes sense, thanks a lot for all the very useful information, I can see what I should be doing now!

[Jack-Works]

People are writing the wrong package.json again and again in the wild. Example: react-hook-form/resolvers#460
I really think this is a problem the TS team should take care of if TS is really serious about fixing the ecosystem.

[Jack-Works]

new example: krisk/Fuse#692

[aleclarson]

I think TypeScript should warn when a types condition exists alongside both MJS and CJS entry points, as it's not really a valid solution in any case. Rather, it just happens to work for MJS importers but not CJS importers.

[justinhelmer]

To add to the dialog:

It took ages to find this solution in order to get dual package exports working properly for TypeScript, but this solution by @Jack-Works works for us 🎉 (with some "gotchas" mentioned below).

Our package.json for our ESM "library" looks like:

{
  "name": "library",
  "type": "module",
  "exports": {
    "./foo": {
      "import": {
        "types": "./dist/foo.d.ts",
        "default": "./dist/foo.js"
      },
      "require": {
        "types": "./dist/foo.d.cts",
        "default": "./dist/foo.umd.cjs"
      }
    },
    "./bar": {
      "import": {
        "types": "./dist/bar.d.ts",
        "default": "./dist/bar.js"
      },
      "require": {
        "types": "./dist/bar.d.cts",
        "default": "./dist/bar.umd.cjs"
      }
    }
  }
}

Having the separate *.d.cts file was the key to getting the library to work for CJS consumers, as mentioned in this thread.

Then, we are able to have our CJS "consumer" use this ESM "library" with the following tsconfig.json:

{
  "compilerOptions": {
    "target": "ES6",
    "module": "NodeNext"
  }
}

We could have also used module=CommonJS in combination with moduleResolution=NodeNext. Using moduleResolution=NodeNext (which is the default when module=NodeNext) is necessary because our "library" package only defines an exports field and not a main. This is because we are using subpath exports to export multiple entry points, and don't need to support older versions of Node.

Here is the kicker: Simply copying the *.d.ts file and renaming it to *.d.cts as mentioned by @Jack-Works was not sufficient for our case. This is because our "library" is an ESM package with relative path references, e.g.:

// @file foo.ts
export { util } from './util.js';

Note the .js extension - this is because ESM requires file extensions for relative paths, and TypeScript does not rewrite paths from .ts to .(m/c)js.

When foo.ts is compiled with TypeScript (and emits declarations), the foo.d.ts file (and the copied foo.d.cts file) will look identical. Then, when our CJS "package" tries to compile our "library" using TypeScript, it sees the .js extension and is not clever enough to treat it as CommonJS. It results in the following error:

"The current file is a CommonJS module whose imports will produce 'require' calls; however, the referenced file is
an ECMAScript module and cannot be imported with 'require'. Consider writing a dynamic 'import("./util.js")'
call instead."

The fix: rewrite all .js extensions in import and export references in our *.d.cts files to instead be .cjs. That means we have a script that runs after build that looks like this:

import { readFileSync, writeFileSync } from 'node:fs';

import { globby } from 'globby';

const paths = await globby(['dist/**/*.ts']);

for (const esmPath of paths) {
  const esmContent = readFileSync(esmPath, { encoding: 'utf8' });

  // Fix file extensions
  const cjsPath = esmPath.replace('.d.ts', '.d.cts');

  // Fix import/export references
  const cjsContent = esmContent.replace(/(.+ from )'(.+).js'/g, "$1'$2.cjs'");

  writeFileSync(cjsPath, cjsContent);
}

This feels very hacky and we would absolutely like to ditch this completely - but it seems to be the only way to achieve the following:

• Writing a pure ESM library using TypeScript that aligns with the NodeJS standards
• Using conditional exports to support both CJS and ESM consumers
• Using subpath exports that work for CJS and ESM consumers

We would ❤️ to see more native support that doesn't require so much juggling.

[kerambit]

Got same problem here

Provided solution works fine, but be sure that consumer code that using your library is in CJS

Forcing CJS target in consumer code looks like a little strange because docs said that choosing nodenext is the only right option to void such problems and nodenext supports CJS libs =/