Unit test runs very slow (6 sec per a test)

[kidqn]

• Issue

After config ts-jest to create-react app successfully, I saw a test case run 6s.
That's very slow. My jest config following the document: (ts|tsx)": "/node_modules/ts-jest/preprocessor.js"

My code

describe("App", () => {
    it("renders only 1 App component", () => {
        const component = shallow(<App />);
        console.log(component.html())
        expect(component).toHaveLength(1);
    });
});

Result: PASS src/containers/test/App.spec.tsx (6.119s)

• Expected behavior

I searched some tutorials on the internet, there is a preprocessor.js file:

const tsc = require("typescript");
const tsConfig = require("./tsconfig.json");

module.exports = {
  process(src, path) {
    if (path.endsWith(".ts") || path.endsWith(".tsx")) {
      return tsc.transpile(src, tsConfig.compilerOptions, path, []);
    } else if (path.endsWith(".js") ) {
      return tsc.transpile(src, tsConfig.compilerOptions, path, []);
    }
    return src;
  }
};

If I use this file, the test case runs on 3 -4s.
Can you explain me this problem? Does preprocessor.js of TS-Jest have more functionalities than that file as well?

[kulshekhar]

I can't comment on this without seeing what setup you're using. Please create a minimal repo with this issue

[clayne11]

I'm experiencing this as well. Updating from ts-jest@20.0.4 -> ts-jest@20.0.7 made my test time go from ~15s -> ~70s.

[Tallyb]

I am using it with angular-jest preset which uses ts-jest. You may want to use the example there are a minimal repo.
https://github.com/thymikee/jest-preset-angular/tree/master/example
Upgrading to TS 2.4.1 seem to speed up things. Anyone else would like to try this?

[kryops]

I am also seeing a substantial increase in test time between 20.0.4 and 20.0.7.

• Node 6.11.1
• TypeScript 2.4.2
• Windows 10
• 84 tests in 25 suites, 40 snapshots - most of the tests are fairly heavy, lots of React application mounting through enzyme

ts-jest 20.0.4: 30-35s
ts-jest 20.0.7: ~60s

It's sadly not open source, the only thing I am able to provide is our jest.json:

jest.json

{
    "setupTestFrameworkScriptFile": "<rootDir>/node_modules/project-watchtower/lib/test/setup-tests.js",
    "transform": {
        ".tsx?": "<rootDir>/node_modules/ts-jest/preprocessor.js"
    },
    "transformIgnorePatterns": [
        "node_modules/(?!swm-component-library)"
    ],
    "testRegex": "\\.(spec|test)\\.tsx?$",
    "moduleFileExtensions": [
        "ts",
        "tsx",
        "js",
        "json"
    ],
    "moduleDirectories": [
        "node_modules"
    ],
    "modulePaths": [
        "<rootDir>/common",
        "<rootDir>/app",
        "<rootDir>"
    ],
    "moduleNameMapper": {
        "\\.(s?css|png|svg|jpg)$": "<rootDir>/node_modules/project-watchtower/lib/test/test-mapper.js",
        "^bundle-loader": "<rootDir>/node_modules/project-watchtower/lib/test/bundle-loader-mapper.js"
    },
    "snapshotSerializers": [
      "enzyme-to-json/serializer"
    ],
    "verbose": false,
    "collectCoverageFrom": [
        "**/*.{ts,tsx}",
        "!**/*.d.ts",
        "!**/node_modules/**",
        "!config/**"
    ],
    "coverageReporters": ["json", "lcov", "text", "teamcity"],
    "testResultsProcessor": "jest-teamcity-reporter",
    "rootDir": "../../../.."
}

[hermanbanken]

Using the node-inspector I've found a potential reason for the slowness:

Every time a Error is created the stacktrace is prepared and this retriggers the Typescript transpile step. Weird, as Typescript already compiled. What causes this I don't know. Does ts-lint manually patch some of these methods to inject the TS compile step? And if so, doesn't that need to use some sort of cache?

[hermanbanken]

Sample project: https://github.com/hermanbanken/ts-jest-bugfix

[hermanbanken]

CPU-20170804T145819.cpuprofile.zip
which you can load in Chrome's Inspector Profiler-tab

[GeeWee]

I think this is due to the fact that our sourcemap-support recompiles the typescript to make the line numbers match in stack traces.

We should be able to cache the files though, shouldn't we @kulshekhar ?

[kulshekhar]

We should be able to cache the files though, shouldn't we

I'm not sure but I think this might be possible

[GeeWee]

#292 is low hanging fruit - but I actually think we might be able to cache better than that - we could cache the files the first time we transpile them (e.g.) in the preprocessor, and then just look up the transpiled files in the source-map-support plugin.

This of course, assumes that source-map-support has access to our preprocessor, even after the preprocess step.

[kulshekhar]

Wouldn't jest handle the caching part if nothing (source code/tsconfig/jest config) has changed?

[GeeWee]

I'm not actually sure. Anyways, let's take a look at the performance after merging in #292, and then see if we actually need to do more.

[kulshekhar]

@kidqn @clayne11 @kryops @hermanbanken Could you please test with 20.0.10 and share your results.

[hermanbanken]

See that #292 is merged 🎉

Will test it in the morning too with my actual project. Good work 👍

[kompot]

Tested 20.0.10 - no visible improvement. Rather small test environment of 65 tests runs 18-20 seconds.
7 of 11 suites comprising those 65 tests took longer than 5 seconds.

[kryops]

wow, this really made our tests a lot faster!

Test Suites: 27 passed, 27 total
Tests:       105 passed, 105 total
Snapshots:   42 passed, 42 total

ts-jest@20.0.4: 35-40s
ts-jest@20.0.7: 90-100s
ts-jest@20.0.10: 20-25s

Thanks so much!

[hermanbanken]

ts-jest@20.0.7:

    ✓ creates many Timeout errors (1995ms)
    ✓ throwing 1000 Errors (5042ms)
    ✓ throwing 1000 Errors, not touching Error.stack (4ms)

ts-jest@20.0.10:

    ✓ creates many Timeout errors (325ms)
    ✓ throwing 1000 Errors (355ms)
    ✓ throwing 1000 Errors, not touching Error.stack (5ms)

🎉

For my real project the difference is visible too, however, the difference is smaller. The time during which our tests run is indeed much shorter though. Setting Error.stackTraceLimit = 0; somewhere early on gives an even greater improvement, as even the first compile during the test runs is skipped I think. Is the cache in #292 specific to source-map-support, or is it the cache from the first compilation that also generated the tests to be run? If it's the first, that would explain why Error.stackTraceLimit = 0; is still faster. Anyway, good work!

[kulshekhar]

Tested 20.0.10 - no visible improvement. Rather small test environment of 65 tests runs 18-20 seconds.
7 of 11 suites comprising those 65 tests took longer than 5 seconds.

@kompot would it be possible for you to create a sample repo with one of the long running tests?

wow, this really made our tests a lot faster!

@kryops awesome!

Is the cache in #292 specific to source-map-support, or is it the cache from the first compilation that also generated the tests to be run?

@hermanbanken That cache is specific to source-map-support. The cache from the first compilation is handled by jest and not by ts-jest.

Anyway, good work!

Thanks. You deserve the bulk of the credit for this though. The repo and the profile were immensely helpful.

At this point, I think any significant speed improvement beyond this might have to come from Jest itself

[GeeWee]

@kompot can you try compiling the tests with tsc and then just running regular jest (i.e. without ts-jest) on them? It'd be interesting to see whether this is just jest being a little tardy or our problem.

[kompot]

Well, seems like the problem was with jest-webpack-resolver
https://github.com/mkg0/jest-webpack-resolver
bumped its' dependencies (from webpack resolve 2.6.x to 3.4.x) and test run time is cut almost in half (11 instead of 18-20sec).

[karlhorky]

@ahnpnl has this issue been addressed? The inactivity doesn't indicate that there is no problem here...

[ahnpnl]

this has been addressed in #1115 , possible related to #908

[Karql]

@ahnpnl imo changing default for isolationModule is not fix for this.
Please look once again at my comment #259 (comment)

[ahnpnl]

I agree changing isolatedModules true is just a workaround. For clean keeping I closed this issue because it’s a known issue after 23.10 and has been addressed.

[danielkcz]

@ahnpnl Has been addressed by a workaround 😆? If there is an existing issue that tracks performance after 23.10, can you link it here, please? Otherwise, I don't follow either why to close this one. Plenty of valuable feedback here.

[ahnpnl]

No I don’t mean addresses by a workaround. I mean workaround is isolatedModule=true. By default, isolatedModule=false which causes the issue. Since this has been linked to the other issue as related area (performance after 23.10, I don’t see any issues of closing this. You can always look back from the link issue

[ahnpnl]

Hi all, would you guys help testing with the new changes mentioned in #1310 (comment) ?

[ahnpnl]

FYI: #1549 will be in alpha version of ts-jest (possibly today). Anyone who is using ts-jest please help to test the alpha version and give us some feedbacks for #1115

[ifeltsweet]

We found that by having a separate tsconfig.json for testing where include: [] is just an empty array improves performance by about 30-40% in our small test suite.

[ahnpnl]

Yes that is exactly what I implemented for alpha release version

[mankinchi]

I'm still facing the issue now. A very simple test of checking 1+1 equals 2 takes 6s. Doing the --maxWorkers=1 trick seems helping a lot, reducing the time to 0.5s.

[PatrykMilewski]

If you wish, you can use this workaround in jest.config.js:

module.exports = {
  transform: {
    "^.+\\.[t|j]sx?$": "babel-jest"
  },
  maxWorkers: 1
};

Reduced complex tests time from 7 seconds to 2.7 seconds and times are much more consistent.

[ahnpnl]

or using isolatedModules: true

[dpinol]

"^.+\.[t|j]sx?$": "babel-jest"

in my case, using babel-jest instead of ts-jest, which is what I had, reports errors in the lines with typescript specific syntax

[Masamei]

We found that by having a separate tsconfig.json for testing where include: [] is just an empty array improves performance by about 30-40% in our small test suite.

This helped me a lot. Reduced the running time on very large test suite by 15%. its awsome :) Thank you guys!

[Bessonov]

My observations (and I don't know much about jest internals or ts-jest internals, so take with a grain of salt):

• jest workers run independently of each worker. This results in:
• Every jest worker compiles typescript on each own. 8 workers = 8 compilations in parallel. Therefore tests are very slow.
• On second run, cache should kick in, but well, it seems like it doesn't happens. Therefore multiple executions are still slow. Probably a bug.
• runInBand or maxWorkers: 1 seems like allowing re-usage of compiled artifacts, because there is only one worker (and caches compiled artifacts in memory?). Therefore, the compilation of first testsuit is still slow (but faster than with multiple compilations in parallel), but following testsuits are very fast and, probably, compiles needed files incrementally.
• isolatedModules turns type checking off. Therefore compiler do less work and the tests are faster, especially if run in parallel.

All in one, I try following workaround now:

• Use isolatedModules in development for speed
• Use maxWorkers: 1 in CI to ensure types

I hope that the issue with caching can be pinned down and resolved.

@ahnpnl can you please reopen this issue?

Related, but I don't think that it is the same: #1310
Unrelated: #1115

[ahnpnl]

Your observation is correct.

Caching is Jest itself does based on cache key which ts-jest provides. ts-jest cannot do anything about caching previous test runs.

[cookernetes]

I am having the same problem with ts-jest:

They are the simple tests that I am running, yet it is taking this long:

[sir-captainmorgan21]

@bbugh I appreciate your solution! However, setting maxWorkers to 1 actually made our unit tests much much slower. We have a library that consists of 115 icon svg components, and it runs it one at a time, taking forever. Also these are Angular projects

[flolu]

Install SWC with

pnpm add -D @swc/core @swc/jest

And add this to your jest.config.js

{
  "transform": {
    "^.+\\.(t|j)sx?$": "@swc/jest"
  }
}

[Bessonov]

@flolu thanks for the great example. AFAIK swc doesn't check types either, so there shouldn't be much difference between swc and isolatedModules: true. Do you have any numbers?

[flolu]

@Bessonov you are right, it is not a huge difference. But it's almost twice as fast in many cases

[DasGandlaf]

Sadly, swc does not work in angular projects. And I also see very slow test times in simple tests. Is there not anything one can do about this? Even just go to an older jest version or something?

[Luxiorawa]

Same issue here, i'm using it on a Nest.js project, but it still use ts-jest under the hood so.. Even basic tests like @cookernetes have shown takes 20 seconds !

I'm currently using the 29.0.3 version