fix: allow Node16/NodeNext/Bundler moduleResolution in project's tsconfig

[akwodkiewicz]

This small change fixes an issue that makes using Jest impossible inside a TypeScript project configured with NodeNext, Node16 or Bundler moduleResolution setting if a Jest configuration file is written in TS.

The default behaviour of ts-node is to read the default tsconfig.json file from the project.
With a hardcoded option of "module: CommonJs" that is used to read the jest.config.ts file, the TypeScript compiler will throw an error, because the modern moduleResolution options are not compatible with
the hardcoded "CommonJs" module value.

The only way to use Jest in such a repo is to change the jest.config.ts file into a JS one (or pass the options in a different way), so that the code responsible of instantiating ts-node is not invoked.

This commit fixes the issue by providing a missing complementary option, moduleResolution: Node, which will work perfectly with module: CommonJs and will not be overridden by any project-specific value in tsconfig.json.

Closes #14740

EDIT
The PR does not close the issue #13350!

This means that the fix works only for jest.config.ts files that do not reference other files using .js extensions. See #14739 (comment)

[netlify]

✅ Deploy Preview for jestjs ready!

Built without sensitive environment variables

Name	Link
🔨 Latest commit	350f078
🔍 Latest deploy log	https://app.netlify.com/sites/jestjs/deploys/664ba6dcaa559d00085a4630
😎 Deploy Preview	https://deploy-preview-14739--jestjs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[SimenB]

Thanks! Could you add a test as well? An integration test is probably the thing here

[akwodkiewicz]

Sure thing, I've never done it before in this project, but I'll try to deliver it in the upcoming days.

[akwodkiewicz]

I have a hard time writing those integration tests. I spent two days already, without any success.

First I tried to create a e2e scenario, but it turned out that the config was not even read when the test invoked the runJest testing util, so I got false positives when running tests before the fix.

Then I realized we most likely have to place the test inside jest-config's test suites, but my test ends with the following error:

Error: Jest: Failed to parse the TypeScript config file __fixtures__/readConfigFileTsNodeCompatibility/bundler/jest.config.ts
  Error: You need to run with a version of node that supports ES Modules in the VM API. See https://jestjs.io/docs/ecmascript-modules
    at readConfigFileAndSetRootDir (/Users/andrzejwodkiewicz/Repos/jest/packages/jest-config/src/readConfigFileAndSetRootDir.ts:43:13)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v18.18.2
 FAIL  packages/jest-config/src/__tests__/readConfigFileTsNodeCompatibility.test.ts
  ● Test suite failed to run

    Jest worker encountered 4 child process exceptions, exceeding retry limit

      at ChildProcessWorker.initialize (packages/jest-worker/build/index.js:804:21)

The idea was to create the actual folders representing various flavours of projects that trigger the bug and place them inside a new __fixtures__ folder:

pwd
/Users/andrzejwodkiewicz/Repos/jest/packages/jest-config/src/__tests__

tree
.
├── Defaults.test.ts
├── __fixtures__
│   └── readConfigFileTsNodeCompatibility
│       ├── bundler                             // <- case 1
│       │   ├── jest.config.ts
│       │   ├── package.json
│       │   ├── src
│       │   │   └── test.spec.ts
│       │   └── tsconfig.json
│       ├── node-16                             // <- case 2
│       │   ├── jest.config.ts
│       │   ├── package.json
│       │   ├── src
│       │   │   └── test.spec.ts
│       │   └── tsconfig.json
│       └── node-next                           // <- case 3
│           ├── jest.config.ts
│           ├── package.json
│           ├── src
│           │   └── test.spec.ts
│           └── tsconfig.json
├── __snapshots__
│   └── normalize.test.ts.snap
├── getMaxWorkers.test.ts
├── jest-preset.json
├── normalize.test.ts
├── parseShardPair.test.ts
├── readConfig.test.ts
├── readConfigFileAndSetRootDir.test.ts
├── readConfigFileTsNodeCompatibility.test.ts    // <- my test
├── readConfigs.test.ts
├── readInitialOptions.test.ts
├── resolveConfigPath.test.ts
├── setFromArgv.test.ts
├── stringToBytes.test.ts
└── tsconfig.json

9 directories, 27 files

The test itself:

import path = require('node:path');
import readConfigFileAndSetRootDir from '../readConfigFileAndSetRootDir';

const rootDir = path.join('__fixtures__', 'readConfigFileTsNodeCompatibility');
const moduleResolutionOptions = ['bundler', 'node-16', 'node-next'] as const;

describe('jest is correctly started from using a jest.config.ts file for a project using module/moduleResolution set to', () => {
  test.each(moduleResolutionOptions)('%s', async opt => {
    const readConfig = async () =>
      readConfigFileAndSetRootDir(path.join(rootDir, opt, 'jest.config.ts'));
    expect(readConfig).not.toThrow();
  });
});

[akwodkiewicz]

It is the await import('ts-node') line that throws the ESM error

jest/packages/jest-config/src/readConfigFileAndSetRootDir.ts

Line 115 in 3ed62f0

const tsNode = await import(/* webpackIgnore: true */ 'ts-node');

In all the readConfigFileAndSetRootDir tests there is not a single test that imports a jest.config.ts file. There are mocks of requireOrImportModule, but surely I don't want to mock a ts-node import. I'm stuck.

[akwodkiewicz]

@SimenB, it seems that I'd need to run the tests in the ESM mode, but that is not how all the test suites are configured to run.

Can we merge the PR without an appropriate test suite?

[github-actions]

This PR is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[akwodkiewicz]

@SimenB, would it be possible for you to take a look and suggest how to write an integration test here? Alternatively, make the decision to skip the tests here?

This PR has a solution to an issue that will affect more and more people since TS is now more openly suggesting to stay away from the module: commonjs setting and move to those that cause the #14740 issue

[michaelfaith]

It is the await import('ts-node') line that throws the ESM error

jest/packages/jest-config/src/readConfigFileAndSetRootDir.ts

Line 115 in 3ed62f0

const tsNode = await import(/* webpackIgnore: true */ 'ts-node');

In all the readConfigFileAndSetRootDir tests there is not a single test that imports a jest.config.ts file. There are mocks of requireOrImportModule, but surely I don't want to mock a ts-node import. I'm stuck.

Could you mock it and use require to get ts-node instead, so the tests can be in commonjs?

[akwodkiewicz]

Could you mock it and use require to get ts-node instead, so the tests can be in commonjs?

If by "it" you mean the function that contains the await import(...) -- registerTsNode() -- then by mocking it I'd mock my changes so I don't think it would make a good test. The tsconfig options are defined 1 line below the await import():

jest/packages/jest-config/src/readConfigFileAndSetRootDir.ts

Lines 112 to 124 in 3ed62f0

	async function registerTsNode(): Promise<Service> {
	try {
	// Register TypeScript compiler instance
	const tsNode = await import(/* webpackIgnore: true */ 'ts-node');
	return tsNode.register({
	compilerOptions: {
	module: 'CommonJS',
	moduleResolution: 'Node',
	},
	moduleTypes: {
	'**': 'cjs',
	},
	});

Unless you had something else in mind. Like, mocking just the await import() so it means require()? Is that possible?

[SimenB]

thanks!

would you mind adding an integration test (like the test added in #12397 - maybe a sibling?) and a changelog entry?

EDIT: sorry, I see you asked for help with the test - I'll take a look tomorrow. But I see you tried a unit test - that probably won't work. We need an integration test (i.e. spawn Jest and have it read the TS config file, not via a unit test)

[SimenB]

I tried to apply this fix to the reproduction from #13350 (after running npm upgrade typescript to get 5.x), but it still fails with the same error. I guess this resolves one warning from tsc, but doesn't resolve the underlying issue?

[SimenB]

regardless, I pushed up a test based on #13350 that fails. If this PR is solving a different issue than that, feel free to tweak it to be for that 🙂

You can run just the test to get the full output by doing cd e2e/esm-config/ts-modern then running node ../../../packages/jest/bin/jest.js from there. Once that is passing, the integration test should pass as well 👍

[akwodkiewicz]

Ok, I see what's the issue with my PR.

The problem I wanted to address here is that a really simple, self-contained jest.config.ts file (without any imports) currently cannot be used if the project's tsconfig.json contains a modern moduleResolution. This happens just because ts-node cannot even register the compiler with incorrect compiler configuration, so hardcoding moduleResolution, so that it's not "taken" from the project works fine.

But the #13350 is a variation that adds some complexity, because with a modern moduleResolution valid import statements in a .ts file are those with .js extensions (like in the example: import config from './some-other-file.js). What's more, they are the only vaild imports if it's an ESM project with moduleResolution set to Node16. CJS projects and "moduleResolution": "Bundler" also allow a .js import, so this issue is also relevant there, but for those cases you can always provide an extensionless import (import config from './some-other-file';) that is correct from the TS perspective and does not cause errors with Jest. Of course, this is not a solution to the issue, but I thought it's important to mention.

I'll first try to add a test case that addresses my original problem, and then I'll try to find out what exactly is responsible for the other issue.

[akwodkiewicz]

The issue is about ts-node not being able to properly load ESM files with those .js imports. I tried providing esm: true config to the ts-node registration or do various things with the module/moduleResolution settings, but without success.

But then I found this: TypeStrong/ts-node#1514

All file extensions are supported, both CommonJS and ECMAScript modules

• Use module: NodeNext, esm: true, and experimentalResolver: true. Everything works: cjs, cts, mjs, mts, with or without file extensions in import specifiers, in CommonJS or ESM files

I modified the test case by adding the following part to the e2e/esm-config/ts-modern/tsconfig.json:

 "ts-node": {
    "experimentalResolver": true
  }

This makes the test work -- the .js import is properly translated to a .ts file.

I don't know if we should bake this resolver into Jest, I honestly don't know how it will impact all the existing working cases.

But I'm starting to wonder if it's even a good idea to hardcode any options in the ts-node registration:

jest/packages/jest-config/src/readConfigFileAndSetRootDir.ts

Lines 119 to 127 in 291df58

	return tsNode.register({
	compilerOptions: {
	module: 'CommonJS',
	moduleResolution: 'Node10',
	},
	moduleTypes: {
	'**': 'cjs',
	},
	});

or should we pass nothing and rely on the tsconfig.json of the current directory? Because that would also fix my original issue (#14740).

EDIT: ...unless we want to bake the experimentalResolver setting and do not make other people modify their projects to make Jest work. Then we could actually do this:

    return tsNode.register({
      experimentalResolver: true,
    });

The idea would be that all tsconfig.json compiler settings in the project are respected (actually they seem to be just ignored) and the experimental resolver takes care of the .js imports.

[SimenB]

Happy to try that! The reason why we try to force CJS is that otherwise the register won't intercept any new require calls made by the config file. If we can just support any TS file without passing options that's be awesome

[SimenB]

Took a look at vite - seems they actually bundle the config file, then execute it. Something like that would work as well (and we'd drop the ts-node dep).

https://github.com/vitejs/vite/blob/2b61cc39a986c44d733aba8c23036d9d83667fac/packages/vite/src/node/config.ts#L1073

[akwodkiewicz]

There's one more thing that bugs me -- I cannot make the solution with baked experimentalResolver: true working. It only works if this property is set in the project itself.

[akwodkiewicz]

@SimenB, I suppose all tests need to pass (no flaky suites you are aware of)? 😅

If that's the case then I'd suggest reverting your test case and I'll revert all the additional changes I made, so that we can at least make importless jest.config.ts work with the modern TS moduleResolution. Is that ok?

[akwodkiewicz]

When you were mentioning vite -- was the idea to pre-compile the config file with tsc or did you actually want to use vite?

[SimenB]

Either should work, but just using tsc is probably better than bringing in a huge dep. Not sure how to cleanly output and then load the file(s) without bundling, tho.

[SimenB]

I'm happy to do that in a separate PR tho, if you wanna get the moduleResolution fix in here first for the simple case with no modern imports in the config file itself

[akwodkiewicz]

I rewrote the history and pushed only the minimal changes to resolve #14740 on top of the current main.

I did not keep your commit with the test case for #13350 (for obvious reasons), but it's available under hash cf7082e -- it will be useful for the other PR.

I added the integration test in my last commit.

---

I'm currently tinkering with TS programmatic API to transpile the config on-the-fly, but I'm a little bit worried this might be quite slow. Anyway, if I manage to do anything that looks like it could work, I'll submit a PR.

[SimenB]

thanks!

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.