[Bug]: ts-jest 29.2 breaks dependency transform

[unional]

Version

29.2.3

Steps to reproduce

1. clone https://github.com/repobuddy/repobuddy/tree/ts-jest-29.2-issue
2. pnpm i
3. pnpm build
4. pnpm jest test

Expected behavior

test pass

Actual behavior

    /home/runner/work/repobuddy/repobuddy/packages/jest/ts/tests/chalk.spec.ts:1
    ({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,jest){import { it } from '@jest/globals';
                                                                                      ^^^^^^

    SyntaxError: Cannot use import statement outside a module

      at Runtime.createScriptFromCode (../../node_modules/.pnpm/jest-runtime@29.7.0/node_modules/jest-runtime/build/index.js:1505:14)

Debug log

see above

Additional context

It is working with ts-jest 29.1

the transform config is:

  transform: {
    '^.+\\.(ts|tsx|cts|mts)$': ['ts-jest', [{
      isolatedModules: true,
      useESM: true,
    }]],
    '\\.m?jsx?$': 'jest-esm-transformer-2'
  },
  transformIgnorePatterns: [],

I have also tried to use transformIgnorePatterns but also not working.

Environment

System:
    OS: macOS 14.5
    CPU: (10) arm64 Apple M1 Pro
  Binaries:
    Node: 20.12.2 - ~/.nvm/versions/node/v20.12.2/bin/node
    npm: 10.5.0 - ~/.nvm/versions/node/v20.12.2/bin/npm
    pnpm: 9.5.0 - ~/.nvm/versions/node/v20.12.2/bin/pnpm
    bun: 0.1.6 - ~/.bun/bin/bun
  npmPackages:
    jest: ^29.7.0 => 29.7.0

[ahnpnl]

Hi, your issue is the same as #4464, you can check my explanation here #4464 (comment)

The main reason is that your test is using Node16 and type: "module" in package.json which makes typescript API to transpile codes into ESM syntax. To fix it, you should set module: "CommonJS" for your test

[unional]

um... thanks.

Seems like that needs to be a tsconfig that is specifically for jest, and it should be a breaking change (I will mention that in @repobuddy/jest, and I can create a PR here to update some docs too if you want).

This is because type: module + Node16/NodeNext enables TypeScript project to consume pure ESM package.
And since TypeScript 5.2, dual CJS/ESM package is not possible anymore with only tsc.

It is still possible by building ESM with tsc, and CJS with other tools such as rollup or vite. But those tools doesn't really need a separate tsconfig.

[ahnpnl]

I was thinking about adding a check on type field from the closest package.json to warn users about this kind of scenario. What do you think?

[unional]

As a user, I generally don't like those warnings. 😄

> @repobuddy/jest@4.1.0 test /home/unional/code/repobuddy/repobuddy/packages/jest
> cross-env NODE_OPTIONS=--experimental-vm-modules NODE_OPTIONS=--experimental-import-meta-resolve jest

ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.
ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.
ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.
ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.
ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.
ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.
ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.
ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.
ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.
ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.
ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.
ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.
ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.
ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.
ts-jest[config] (WARN) message TS151001: If you have issues related to imports, you should consider setting `esModuleInterop` to `true` in your TypeScript configuration file (usually `tsconfig.json`). See https://blogs.msdn.microsoft.com/typescript/2018/01/31/announcing-typescript-2-7/#easier-ecmascript-module-interoperability for more information.

 RUNS  ts/presets/ts-watch/jest-preset.spec.ts
 RUNS  ts/presets/watch/jest-preset.spec.ts
 RUNS  ts/presets/ts-esm-watch/jest-preset.spec.ts
 RUNS  ts/presets/ts-esm/jest-preset.spec.ts
 RUNS  ts/presets/ts-cjs/jest-preset.spec.ts
 RUNS  ts/presets/ts-cjs-watch/jest-preset.spec.ts
 RUNS  ts/presets/ts/jest-preset.spec.ts
 RUNS  ts/presets/jsdom-ts-watch/jest-preset.spec.ts
 RUNS  ts/presets/jsdom-ts-esm-watch/jest-preset.spec.ts
 RUNS  ts/presets/jsdom-ts-esm/jest-preset.spec.ts
 RUNS  ts/presets/jsdom-ts-cjs/jest-preset.spec.ts
 RUNS  ts/presets/jsdom-ts-cjs-watch/jest-preset.spec.ts
 RUNS  ts/presets/jsdom-ts/jest-preset.spec.ts
 RUNS  ts/presets/js-esm-watch/jest-preset.spec.ts
 RUNS  ts/presets/js-esm/jest-preset.spec.ts

When I saw it the first time, it may be good information.
But when it shows up multiple times and on every run, it kinda get in the way. 🤣

I would prefer to keep them in a troubleshooting section of readme or the doc

Or if you can intercept the jest Jest encountered an unexpected token error and amend the message when it actually fails (just like the esModuleInterop issue, type: module doesn't guarantee to fail). But I don't think that is possible and is pretty fragile.

[ahnpnl]

Oh it's not possible to intercept Jest encountered an unexpected token because this error is produced from Jest runtime layer, not on transformer layer like ts-jest.

Maybe we update doc to instruct using ESM and troubleshooting page as well of course. How about that?

[unional]

Actually, I'm curious when does the tsc get into the picture.

I have tried adding module: CommonJS in tsconfig.json and like you said it works.

However, that would break the tsserver as it uses tsconfig.json thus now not able to read pure ESM packages.

I tried adding module: CommonJS in the transform of ts-jest and that doesn't work.

Maybe I need to set that in the global setting? (I am away from my computer now, can check that tomorrow)

UPDATE: I guess module: CommonJS in transform wouldn't work. Should use the tsconfig field. It shows up in the autocomplete so... 🫠

[ahnpnl]

Oh I found out your configuration in https://github.com/repobuddy/repobuddy/blob/851bacacfc0fd6828d58af8fddfbf24fa19f9b44/packages/jest/jest.config.mjs#L34 was a bit wrong.

the transformer option should be an object

{
      isolatedModules: true,
      useESM: true,
      diagnostics: {
        // https://github.com/kulshekhar/ts-jest/issues/3820
        ignoreCodes: [151001]
      },
}

but you put as an array

[{
      isolatedModules: true,
      useESM: true,
      diagnostics: {
        // https://github.com/kulshekhar/ts-jest/issues/3820
        ignoreCodes: [151001]
      }
}]

If you correct this, you don't need to change anything about tsconfig

[unional]

Oh I found out your configuration in...

Ar, thanks. Didn't realize that.

On the other hand, I tried doing the module: CommonJS workaround in the consuming project (private). The config it uses is correct (['ts-jest', { ... }] instead of ['ts-jest', [{ ... }]]. The issue about only occurs to the local config inside repobuddy)

It doesn't work because under CommonJS it can't read pure ESM package.

You will get error like this:

Cannot find module 'type-plus' from 'src/utils/define_columns.spec.ts'

I'm using type-plus@8.0.0-beta.2 which is a pure ESM package.

UPDATE: correction, type-plus@8.0.0-beta.2 is not a pure ESM package, it has:

  "exports": {
    ".": {
      "import": {
        "types": "./esm/index.d.ts",
        "default": "./esm/index.js"
      },
      "default": "./esm/index.js"  // <-- this
    },
    "./package.json": "./package.json"
  },

which is not right for CJS build. I think I have that for some browser need but I could be wrong. I was testing this by removing that line directly in node_modules.

Publishing type-plus@8.0.0-beta.3 without that field.

[unional]

Maybe we update doc to instruct using ESM and troubleshooting page as well of course. How about that?

Sounds good.

[ahnpnl]

Do you still have the issue Cannot find module? If yes, would you pls update the branch to have the example to check? Thanks :)

[unional]

Do you still have the issue Cannot find module? If yes, would you pls update the branch to have the example to check? Thanks :)

Yes but it is in a private repo.
I'll create a testcase in repobuddy

[unional]

I tried to create a testcase but I run into ReferenceError: exports is not defined error.
Seems like some code is in CJS or transpiled to CJS and jest is not happy about it.

I saw some closed issues e.g.: #2399
but seems like this is a different case.

It's the test in https://github.com/repobuddy/repobuddy/tree/main/testcases/dual-ts-jest

It can be executed through pnpm dtj test:jest.

The same condition is working when using swc: https://github.com/repobuddy/repobuddy/tree/main/testcases/dual-ts-swc

[MBurchard]

I'm very sorry, but I don't understand this breaking change.
In my configuration I define extremely clearly that TS files are to be regarded as ESM.
ESM uses import.
So why should this suddenly no longer work after a minor version update?

/** @type {import('ts-jest').JestConfigWithTsJest} */
module.exports = {
  preset: 'ts-jest',
  testEnvironment: 'node',
  collectCoverage: true,
  collectCoverageFrom: ['./src/**'],
  coverageProvider: 'v8',
  coverageThreshold: {
    global: {
      lines: 90,
    },
  },
  extensionsToTreatAsEsm: ['.ts'],
  moduleNameMapper: {
    '^(\\.{1,2}/.*)\\.js$': '$1',
  },
  transform: {
    '^.+\\.[tj]s$': [
      'ts-jest',
      {
        diagnostics: {
          ignoreCodes: ['TS151001'],
        },
      },
    ],
  },
};

[unional]

@MBurchard what's the error did you get?
And just in case, you have the experiment flag to tell node handling esm?

Also, what's your TS version and are you using module Node16/NodeNext?

[ahnpnl]

I'm thinking about reverting the support for Node16/NodeNext and leave it for the next major release. That would solve quite some issues.

[MBurchard]

I have a small project: https://github.com/MBurchard/bit-log
It worked well (main branch) until i tried to update the dependencies (chore/update-dependencies).
The error is, as described in this topic: "SyntaxError: Cannot use import statement outside a module"
The experiment flag does not solve the issue, or leads to other errors that were not there before.
I also want to run my test cases from IntelliJ, so a solution that works via ‘jest.config.cjs’ would be much more flexible.

I would also prefer this topic to be moved to the next major release and for there to be a well-documented solution that does not use experimental flags.

[unional]

Confirmed that the exports is not defined error I got is gone with 9.2.4. 😿

[ahnpnl]

Thanks for the feedbacks 🙏🙏