Incremental builds with .tsbuildinfo-files are not faster than non incremental builds

[DavidZidar]

Bug Report

TypeScript version 4.3.2

I am unable to see any measurable performance difference using incremental builds using tsc --noEmit with or without "incremental": true in compilerOptions or the --incremental CLI argument. Builds take the same amount of time before and after the tsconfig.tsbuildinfo is created.

I have tried this in serveral different projects on both Windows and Linux, the results are the same. I also used the performance tracing feature and the flame graphs look pretty much identical with or without the tsbuildinfo file.

One project takes about 14.5 seconds on my machine without incremental build and about 14.5 seconds with incremental.

Another project takes 3.5 seconds on Windows without and 3.5 seconds with. The same project took 3 seconds on Linux without and 3 seconds with.

First flame graph with no tsbuildinfo

Second build with tsbuildinfo present

🔎 Search Terms

incremental
tsbuildinfo
performance

🕗 Version & Regression Information

• I was unable to test this on prior versions because limited time

⏯ Playground Link

Not possible.

💻 Code

N/A

🙁 Actual behavior

Incremental builds perform the same as non incremental builds.

🙂 Expected behavior

I expect incremental builds to be at least a little bit faster.

[DanielRosenwasser]

I have seen similar results with Pyright's codebase (@jakebailey). It seems like TypeScript still ends up doing type-checking and taking the same amount of time when enabling --incremental.

When I brought this up with @sheetalkamat, she mentioned that

TypeScript/src/compiler/builderState.ts

Line 108 in 391f9ff

const symbol = checker.getSymbolAtLocation(importName);

could be the culprit, but we'd need to investigate.

[amcasey]

While it's certainly true that incremental build isn't always faster than clean build (we're working on it), it's not clear to me that that's the issue here. In particular, those two trace (thanks!) do look suspiciously similar. When an incremental build fails to out-perform a clean build, it is generally because the incremental build did different work than the clean build (i.e. it spent too long determining what updates were needed) and not because it did the exact same work.

Neither of the traces in the description appears to show buildinfo emit. That seems strange to me, but I remember there were some changes in that area in 4.3 (lazier and possibly hash-based?). It's possible we're just using hashing because noEmit is set - I can't remember the specific rules.

Any chance you can share one of your projects?

[amcasey]

If you can't share your code, I would find it helpful to see your tsconfig (feel free to censor names from your project) and tsc commands. Also, the output of --extendedDiagnostics, if that's not too much trouble. Thanks!

[DavidZidar]

Thanks for looking into this! I'm sure it has been working before but I can't get it to work in any of my projects any more.

I'm unable to share any of the code, so I hope the following helps.

Here is the tsconfig for the project I used to create the flame graphs, I just removed some of my customers paths and typings:

{
  "compilerOptions": {
    "baseUrl": ".",
    "esModuleInterop": true,
    "experimentalDecorators": true,
    "importHelpers": true,
    "incremental": true,
    "jsx": "react-jsx",
    "lib": [
      "dom",
      "es6"
    ],
    "module": "commonjs",
    "moduleResolution": "node",
    "noUnusedLocals": true,
    "noUnusedParameters": false,
    "paths": {
      "*": [
        "node_modules/*",
        "node_modules/@types/*",
        "typings/*"
      ],
      "Api/*": [
        "Api/*"
      ],
      "ClientModel/*": [
        "ClientModel/*"
      ],
      "PageComponents/*": [
        "PageComponents/*"
      ]
    },
    "skipDefaultLibCheck": true,
    "skipLibCheck": true,
    "strict": true,
    "strictNullChecks": false,
    "suppressImplicitAnyIndexErrors": true,
    "sourceMap": true,
    "target": "es5",
    "types": [
      "bootstrap.native",
      "hammerjs",
      "jasmine",
      "jquery",
      "jquery.validation",
      "jquery-validation-unobtrusive",
      "modernizr",
      "slick-carousel"
    ],
    "typeRoots": [
      "./node_modules/@types/",
      "./typings/"
    ]
  },
  "ts-node": {
    "transpileOnly": true
  }
}

Here's two runs with extended diagnostics of the same project.

> rm *.tsbuildinfo

> tsc --noEmit --extendedDiagnostics
Files:                         831
Lines of Library:            27739
Lines of Definitions:       144641
Lines of TypeScript:         44310
Lines of JavaScript:             0
Lines of JSON:                   0
Lines of Other:                  0
Nodes of Library:           119202
Nodes of Definitions:       416492
Nodes of TypeScript:        216253
Nodes of JavaScript:             0
Nodes of JSON:                   0
Nodes of Other:                  0
Identifiers:                269543
Symbols:                    283949
Types:                       80758
Instantiations:             372446
Memory used:               389408K
Assignability cache size:    47816
Identity cache size:           782
Subtype cache size:           5259
Strict subtype cache size:     697
I/O Read time:               0.12s
Parse time:                  1.33s
ResolveModule time:          0.45s
ResolveTypeReference time:   0.03s
Program time:                2.05s
Bind time:                   0.72s
Check time:                 12.01s
I/O Write time:              0.00s
printTime time:              0.02s
Emit time:                   0.02s
Total time:                 14.79s

> ls *.tsbuildinfo

    Directory: C:\path\to\project

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---          2021-06-02    11:49         144181 tsconfig.tsbuildinfo

> tsc --noEmit --extendedDiagnostics
Files:                         831
Lines of Library:            27739
Lines of Definitions:       144641
Lines of TypeScript:         44310
Lines of JavaScript:             0
Lines of JSON:                   0
Lines of Other:                  0
Nodes of Library:           119202
Nodes of Definitions:       416492
Nodes of TypeScript:        216253
Nodes of JavaScript:             0
Nodes of JSON:                   0
Nodes of Other:                  0
Identifiers:                269543
Symbols:                    283949
Types:                       80758
Instantiations:             372446
Memory used:               373535K
Assignability cache size:    47816
Identity cache size:           782
Subtype cache size:           5259
Strict subtype cache size:     697
I/O Read time:               0.12s
Parse time:                  1.36s
ResolveModule time:          0.46s
ResolveTypeReference time:   0.03s
Program time:                2.11s
Bind time:                   0.75s
Check time:                 11.67s
Total time:                 14.53s

[amcasey]

Some initial observations:

1. Are those path mappings required? Maybe they're so that you can import from "Foo", rather than "./Foo"? I wouldn't expect the first one (i.e. "*") to make a difference though - the compiler should already be looking in node_modules and the type roots.
2. That's a lot of check time for a medium-sized project. I don't see any hot spots in the trace, but it does seem like something's going wrong. The most common explanation I've seen for extra check time spread evenly across a lot of files is having references to two or more versions of a single library (e.g. pulling in react directly and a different version indirectly).
3. With literally no changes between the two builds, I would have expected timestamp checks to prevent checking entirely. I'll see if I can repro that locally.

[sheetalkamat]

Time checks are only done with build and not project build

[amcasey]

@sheetalkamat Ah, because we can't know the list of files to check? What are we checking instead? The hashes of the input files listed in buildinfo?

[DavidZidar]

The path mappings aren't so much importing "Foo" rather than "./Foo" but more like "Foo" rather than "../../../../../Bar/Foo" which is both hard to read and a lot harder to type and different for every file depending how deep it is in the folder hierarchy. I prefer common framework imports to be rooted in a known path.

[amcasey]

I tried the same test with https://github.com/ant-design/ant-design, which is roughly the same size, and the second build stopped after binding (i.e. no check time).

@DavidZidar Do you want to give that a shot locally and confirm that it's specific to your project(s) and not (e.g.) your file system? https://github.com/amcasey/ant-design/tree/GH44305

npm install --ignore-scripts
npx --no-install tsc -p . --noEmit --incremental --extendedDiagnostics

[DavidZidar]

@amcasey I tried but I can't seem to install the dependencies.

npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving: antd@4.15.1
npm ERR! Found: react@17.0.2
npm ERR! node_modules/react
npm ERR!   dev react@"^17.0.1" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer react@"^16.0.0" from @ant-design/hitu@0.0.0-alpha.16
npm ERR! node_modules/@ant-design/hitu
npm ERR!   dev @ant-design/hitu@"^0.0.0-alpha.13" from the root project

However, as I stated originally I have tried to observe a measurable difference on two different machines and in Windows and in Linux (in WSL 2) on a different project on the second machine. But maybe I'm doing the same thing that's causing this issue in all my projects?