CPU usage on idle?

[mszczepanczyk]

I added checker to the build process in our app and after a while I started getting complaints about high CPU usage with webpack --watch. Interestingly only windows people raised it. I've tried to investigate the issue on my macOS and realized the checker processes use quite a lot of CPU resources even when idling:

V8 inspector attached to one of them shows no executions:

I suspect the CPU consumption is caused by GC or file watchers. Any idea how to fix/debug this one?

[piotr-oles]

Could you provide more information about your setup? The best option would be a bug reproduction repository and information about your OS and hardware. If you are not able to create bug reproduction repository, send at least your webpack config :)

[mszczepanczyk]

@piotr-oles hey, thanks for getting back!

I've tried to replicate the problem in a new repo with similar setup, and some auto-generated typescript files but the worst I'm getting so far is ~4% cpu on idle. It kind of feels like the number is slowly getting higher with more ts files but it's hard to tell as it jumps around and my dumb ad hoc generator quickly exceeds available stack size. I couldn't find any benchmark/test files that I could use though.

That said, even 4% on idle isn't nothing, especially once compared to webpack which uses zero when watching.

I'm using current node lts on macOS, cpu i5-7360U (2 cores)

Here's the repo: https://github.com/mszczepanczyk/fork-ts-checker-high-cpu-on-idle

[taylortdixon]

I'm not able to share the repository I am working on, but I am seeing this same issue, where it's getting up to 40% on idle.

System Information:
Microsoft Windows 10 Pro 10.0.14393 Build 14393
i7-6820HQ 2.70GHz 4 Cores

Dependencies:
fork-ts-checker-webpack-plugin: 1.0.1
Typescript 3.4.1
Webpack 4.29.6
happypack: 5.0.1
Ts-Loader 5.3.3

[johnnyreilly]

As part of your tweaking I suggest dropping happypack and starting to use the incremental API. Check out the config here for inspiration: https://github.com/TypeStrong/ts-loader/blob/master/examples/fork-ts-checker-webpack-plugin/webpack.config.development.js

Stopping using happypack might reduce the resource usage significantly

[delaaxe]

I have the exact same issue on Windows 10 using this via create react app v2.1.8

[ajkettun]

Same issue here (Win 10). Upgraded from 0.4.6 to 1.0.1. 1.0.1 uses about 10% CPU, 0.4.6 practically none.

[NeKJ]

@johnnyreilly

I have the same issue with my system:

Windows 10 1809 x64
Node 10.15.1
Typescript 3.4.2
Gulp 4.0.0
ts-loader 5.3.3
webpack 4.29.6

I just tried your suggestion and changed to ts-loader to transpileOnly with the useTypescriptIncrementalAPI enabled:

But unfortunately the issue remains, fork-ts-checker plugin eats 5-8% cpu (of total 4 real i7 cores) while sitting idle:

If I set instead the useTypescriptIncrementalAPI to false, then the cpu idle usage drops to 0. Therefore this is issue definitely related with this new setting.

[0xorial]

I think I had some improvement when I tried setting environmental variable TSC_NONPOLLING_WATCHER = '1'.
More info

[NeKJ]

I read the link and figured out that this is the correct setting to use:

set TSC_WATCHFILE=UseFsEvents

I tested it and indeed cpu usage on idle is 0.0% without any issues (so far) watching and recompiling the files upon change.

[johnnyreilly]

Is this PR-able? I'd be interested in baking in better default behaviour if we can.

[NeKJ]

Yeah I guess so. Probably we could also bake in a new option too, that you could switch from polling to fsevents.

[johnnyreilly]

If you'd like to try something out we'd be very grateful ❤️

[NeKJ]

sure :)

[NeKJ]

PR ready, check it out :)
#256

[harrybin]

Hi,
I also have this issue. In my real-world project the node process eats over 60% cpu.
First I thought it was an issue of the webpack-dev-server: webpack/webpack-dev-server#1846
But after "evilebottnawi" said it must be a loader or plugin I was able to drag it down to fork-ts-checker-webpack-plugin.
The issue I created for webpack-dev-server (webpack/webpack-dev-server#1846) also contains a simple hello world repro project. That project only causes 2-5% CPU, but already shows the issue: all d.ts file in node_modules are loaded again and again.
When adding a large lib to node_modules (e.g. @material-ui/core and @material-ui/icons) to increase the number of d.ts file in node_modules you can raise the CPU usage dramatically.

set TSC_WATCHFILE=UseFsEvents works for me too

[johnnyreilly]

To spread the knowledge about this issue (and the workaround) I've blogged it here: https://blog.johnnyreilly.com/2019/05/typescript-and-high-cpu-usage-watch.html

Hope it helps people reading this!

[allnash]

When will this be fixed? it is eating my battery.

[johnnyreilly]

Please see the blog post linked above for details @allnash. This is not an issue with the fork-ts-checker-webpack-plugin but with the watch implementation in TypeScript. We're not planning to workaround it in the plugin itself, but there is advice in the blogpost which will allow you to work around this on your own build.

[jasonwilliams]

@johnnyreilly you are a hero, thanks for chasing this up, i'm late to the party but this has bugged me for a while, great blog post, I've managed to fix it by adding TSC_WATCHFILE=UseFsEvents.

I've found UseFsEvents is the best option, UseFsEventsWithFallbackDynamicPolling was still using quite a bit of CPU (maybe trying to poll?)

If you're using a Mac (even with a docker guest) you don't need polling-fallback, only with a windows host.

In case anyone else comes here, Typescript's reasoning is layed out here:
https://github.com/microsoft/TypeScript-Handbook/blob/master/pages/Configuring%20Watch.md#background

[activescott]

FYI I added ENV TSC_WATCHFILE UseFsEvents to a Dockerfile that I was noticing exceptionally high cpu usage with and that fixed it! Thanks for this!

This was happening with Next.js which ends up running this service in dev mode.

[samupl]

Unfortunately, that did not help in my case. I tried both UseFsEvents and UseFsEventsWithFallbackDynamicPolling and I still get 99% CPU usage on idle.

In my use-case, I use a node:13-alpine docker image with ENV TSC_WATCHFILE=UseFsEvents set, and the app code mounted from the host directory. For reference, here's my Dockerfile:

LABEL maintainer="PIRC.PL"

ENV TSC_WATCHFILE=UseFsEvents
RUN mkdir /app && chown -R node /app

ADD --chown=node package.json /app/
ADD --chown=node yarn.lock /app/

WORKDIR /app
USER node

RUN yarn install

ADD --chown=node . /app/
RUN chmod +x /app/docker-entrypoint.sh
ENTRYPOINT ["/app/docker-entrypoint.sh"]

The entrypoint simply runs yarn dev. The app is a Nuxt.JS app with typescript support. I can see the /app/node_modules/fork-ts-checker-webpack-plugin/lib/service.js script to take 100% cpu.

[piotr-oles]

In order to fs-events to work, you need pre-compiled binaries to be fetched from the npm. These binaries are different for each node version. Please make sure the fs-events have been successfully installed inside a docker container. If not, you can try to fix it by removing yarn.lock :)

[remorses]

I got a JavaScript heap out of memory error on mac, caused by nextjs dev server, it wasn't even running when it happend
previous days there were many node processes taking 100% cpu
As you can see the running process was from node_modules/fork-ts-checker-webpack-plugin/lib/service.js

{
  "header": {
    "reportVersion": 1,
    "event": "Allocation failed - JavaScript heap out of memory",
    "trigger": "FatalError",
    "filename": "report.20200408.150803.13103.0.001.json",
    "dumpEventTime": "2020-04-08T15:08:03Z",
    "dumpEventTimeStamp": "1586351283778",
    "processId": 13103,
    "cwd": "/Users/morse/Documents/GitHub/nanoservices/frontend",
    "commandLine": [
      "/Users/morse/.nvm/versions/node/v12.13.0/bin/node",
      "--max-old-space-size=2048",
      "/Users/morse/Documents/GitHub/nanoservices/node_modules/fork-ts-checker-webpack-plugin/lib/service.js"
    ],
    "nodejsVersion": "v12.13.0",
    "wordSize": 64,
    "arch": "x64",
    "platform": "darwin",
    "componentVersions": {
      "node": "12.13.0",
      "v8": "7.7.299.13-node.12",
      "uv": "1.32.0",
      "zlib": "1.2.11",
      "brotli": "1.0.7",
      "ares": "1.15.0",
      "modules": "72",
      "nghttp2": "1.39.2",
      "napi": "5",
      "llhttp": "1.1.4",
      "http_parser": "2.8.0",
      "openssl": "1.1.1d",
      "cldr": "35.1",
      "icu": "64.2",
      "tz": "2019a",
      "unicode": "12.1"
    },
    "release": {
      "name": "node",
      "lts": "Erbium",
      "headersUrl": "https://nodejs.org/download/release/v12.13.0/node-v12.13.0-headers.tar.gz",
      "sourceUrl": "https://nodejs.org/download/release/v12.13.0/node-v12.13.0.tar.gz"
    },
    "osName": "Darwin",
    "osRelease": "19.2.0",
    "osVersion": "Darwin Kernel Version 19.2.0: Sat Nov  9 03:47:04 PST 2019; root:xnu-6153.61.1~20/RELEASE_X86_64",
    "osMachine": "x86_64",
...

[piotr-oles]

@mszczepanczyk , @taylortdixon , @delaaxe , @ajkettun , @NeKJ , @harrybin , @allnash , @jasonwilliams , @activescott , @samupl , @remorses
Please try fork-ts-checker-webpack-plugin@alpha - I've published a new version which should resolve this issue 🚀(now the plugin uses webpack's watcher instead of typescript's one)
I will close this issue to clean-up the backlog. If this release didn't solve the issue, we can re-open this :)

[francisu]

See this for problems with the 4.x line of this plugin: #655