[Feature]: Allow using tsx instead of ts-node for loading TS config files

[MasterOdin]

🚀 Feature Proposal

I would like to be able to use the tsx compiler for handling the jest.config.ts file, as opposed to relying on ts-node.

Motivation

tsx is a quickly rising alternative to ts-node as it runs much faster as it runs on the back of esbuild, and doesn't do typechecking. While parsing the jest configuration file should already be generally quick, it could be made quicker to using tsx. Additionally, for teams that have moved to tsx, it would allow having just one runtime compiler in their dependency for everything, vs having tsx for most things, and ts-node just for reading jest.config.ts files.

Example

It would be used implicitly within jest-config library where when it detects a file that ends with .ts, it checks if ts-node is available and uses that, else it'll check for tsx and use that. If neither are available output an error message that neither could be found.

All the user would need to do would be to have tsx installed via their package manager and available via node_modules.

Pitch

Looking at https://github.com/facebook/jest/blob/main/packages/jest-config/src/readConfigFileAndSetRootDir.ts, there didn't seem to be an obvious way to modify the behavior of how it loads config files, where even if it was possible to get tsx registered before triggering jest-config, it would still attempt to use ts-node for .ts extension.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[MasterOdin]

I'm still interested in this. jest is the only thing pulling in ts-node in our stack, and it would be nice to be able to eliminate it for tsx.

[SimenB]

Making this pluggable would be nice - we had a PR at some point for an SWC-based loader as well.

Thoughts on adding some sort of loader docblock at the top of the file, similar to e.g. /** @jest-environment jsdom */?

[MasterOdin]

Sure, I think that adding a brief loader docblock at the top of the file would make sense, in the concept that within your dependency tree you end up with multiple TS loaders for whatever reason, and you want to make sure that you end up using a specific one, vs the default ordering of jest. Adding support for that should be pretty straight-forward, as compared to actually implementing these other loaders.

we had a PR at some point for an SWC-based loader as well.

Do you know what it was named? I did find a PR for esbuild-loader (#12041) that has stalled, but not swc.

[jsardev]

I think we could use something like a require option where you could put whatever transpiler out there. This is how ava does it. This way you can use whatever you want: @swc-node/register, ts-node/register, esbuild-register.

Edit: My bad, this is working on running the tests, not loading the config file

[LinusU]

I was trying to do this today and thought it would be as easy as adding --loader @esbuild-kit/esm-loader to NODE_OPTIONS:

NODE_OPTIONS='--experimental-vm-modules --loader @esbuild-kit/esm-loader' jest

However, when I do that I get an interesting error:

TypeError: Unexpected response from worker: undefined
    at ChildProcessWorker._onMessage (/Users/linus/my-project/node_modules/jest-worker/build/workers/ChildProcessWorker.js:289:15)
    at ChildProcess.emit (node:events:527:28)
    at emit (node:internal/child_process:938:14)
    at processTicksAndRejections (node:internal/process/task_queues:84:21)

Adding some logging to node_modules/jest-worker/build/workers/ChildProcessWorker.js reveals that this is the message being sent from the child process:

{
  "type": "dependency",
  "path": "file:///Users/linus/my-project/node_modules/jest-worker/build/workers/processChild.js"
}

It seems like ChildProcessWorker is expecting an array where the first item is the message type, not an object though...

I'm not really sure how to continue debugging this, but would love some input!

Generally, I think it would be amazing if Jest could work with the --loader argument to Node.js!

[LinusU]

Okay, turns out the mysterious object above wasn't emitted from Jest at all, it was the loader.

Filed an issue about that here: esbuild-kit/esm-loader#43

Knowing that I simply added this code to ignore the messages:

  }
  _onMessage(response) {
+   // Ignore messages emitted by @esbuild-kit/esm-loader
+   // ref: https://github.com/esbuild-kit/esm-loader/issues/43
+   if (typeof response === 'object' && typeof response.type === 'string' && response.type === 'dependency') {
+     return
+   }
    // TODO: Add appropriate type check
    let error;
    switch (response[0]) {

This led me to the next problem.

SyntaxError: Unexpected token, expected "{"

Well, this stack trace was in Babel, and I didn't want to use that. So I added "transform": {} to my Jest config, and tried again:

SyntaxError: Unexpected token ':'

This time the stack trace only shows Runtime.loadEsmModule.

Going in to this function it seems like it always loads the file from disk and passes it thru a function that calls the transformers. I tried replacing it with just an import of the file instead, something like:

        return core;
      }
-     const transformedCode = await this.transformFileAsync(modulePath, {
-       isInternalModule: false,
-       supportsDynamicImport: true,
-       supportsExportNamespaceFrom: true,
-       supportsStaticESM: true,
-       supportsTopLevelAwait: true
-     });
+     const transformedCode = `import '${modulePath}'`;
      try {
        const module = new (_vm().SourceTextModule)(transformedCode, {
          context,

This results in the following error though:

● Test suite failed to run

Your test suite must contain at least one test.

Hmm, alright, I'm not sure why it gives that specific error, but SourceTextModule won't be able to route import calls thru the specified loader as far as I can tell. (actually, I'm not sure about this anymore, I guessed that since it looked like imports were routed via the importModuleDynamically callback, but I see now that that is just when calling import(...))

Lets just try calling my loader here directly and see if it works:

        return core;
      }
-     const transformedCode = await this.transformFileAsync(modulePath, {
-       isInternalModule: false,
-       supportsDynamicImport: true,
-       supportsExportNamespaceFrom: true,
-       supportsStaticESM: true,
-       supportsTopLevelAwait: true
-     });
+     const esmLoader = await import('@esbuild-kit/esm-loader')
+     const transformedCode = (await esmLoader.load(`file://${modulePath}`, { format: 'esm' }, (url) => {
+       return { source: fs().readFileSync(new URL(url), 'utf8') }
+     })).source
      try {
        const module = new (_vm().SourceTextModule)(transformedCode, {
          context,

Test Suites: 15 passed, 15 total
Tests: 53 passed, 53 total

Success!

Okay, if there is a way to get ahold of the loader some way I think using it to load files would be viable.

@SimenB would there be any interest in maybe adding a flag for Jest that makes it use the loader that Node.js uses to load test files? I think that this would be awesome since we can run the tests in the same way that we run the normal code!

---

edit: actually, adding a log of transformedCode shows that it has import statements in it, that in my case refers to TypeScript files. So I'm actually only transforming the first file myself withe the call to esmLoader.load, any imported files seems to be transformed by Node.js using the loader specified with --loader.

edit2: hmm, no every file is passed thru here I think, via the link function of the module...

[SimenB]

I'd be interested in a PR that shows the changes needed. 👍

[LinusU]

Here it is! 🎉

#13521

[MasterOdin]

#13521 wouldn't address the issue of how jest loads the config file though, just how it transforms and runs the test files right?

[LinusU]

@MasterOdin it seems like you are right, I will make a small update to address that 👍

In fact, I didn't notice that this issue was talking about config file specifically at all 😅

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[LinusU]

(not stale)

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[LinusU]

(not stale)

[SimenB]

Still happy to take a PR adding support for some sort of docblock to the config file if anyone's up for it in the new year 😀

[MasterOdin]

@SimenB Trying to find time to do this now, but to be clear on your comments on this thread as well as in #12041 and #11989 is that if the user does not specify a docblock, then jest will only try to use ts-node, throwing the error (plus perhaps linking to the docs about the new docblock) if that's not installed. The only way to use an alternative loader is by specifying the docblock, e.g. /* @jest-config-loader tsx */, or whatever. Of course, if that loader is not installed, then it'll throw the same error as missing ts-node, just mentioning the specified package.

[SimenB]

Yeah, that sounds about right. 👍 We can probably remove built-in ts-node in next major and have that also be opt-in via a docblock.

[SimenB]

Also, not sure if we should define som "config loader" interface (just path-to-config.* -> Config.InitialOptions) rather than us instantiating e.g. tsx and having to provide it options. Us just doing const config = import(moduleNameFromDocblock).then(m => m.default(pathToConfig)) seems better than having to pass a bunch of different options depending on what the "config loader" does. Then we could link to modules providing this interface for whatever module you wanna use.

[LinusU]

@SimenB have you considered the approach in #13521? Instead of having anything specific to Jest, that would then work with any Node.js compatible loader. It would also work with all code, instead of just the config files.

[Kurt-von-Laven]

ts-node hasn't seen a release in over a year at this point and still doesn't support TypeScript 5.0's multiple inheritance feature for tsconfig.json files. tsx, swc-node, or the native Node.js ESM loader are looking increasingly like the way of the future to me.

we had a PR at some point for an SWC-based loader as well.

Do you know what it was named? I did find a PR for esbuild-loader (#12041) that has stalled, but not swc.

@MasterOdin, I believe it was #13779.

[adanski]

It's worth to mention ts-node does not work with the newest Node 18.x and 20.x anymore (works only as a loader but not a standalone command) which might make it pretty much useless for anything other than Jest config file.

TypeStrong/ts-node#1997
TypeStrong/ts-node#2094

[peterbe]

Any news on this? Or workarounds?

As far as I understand, the problem is that ts-node can't handle what tsx can handle.

Also, is this problem with jest or with ts-jest?

[axmad386]

Any news on this? Or workarounds?

As far as I understand, the problem is that ts-node can't handle what tsx can handle.

Also, is this problem with jest or with ts-jest?

Sadly, I give up with jest to handle typescript and esm. Migrated to vitest solve all the problems.