Try migrating off of tsb and using normal tsc for building VS Code

[mjbvz]

VS Code currently use the gulp-tsb library for building VS Code. This library has some capability problems with some typescript features (jrieken/gulp-tsb#81).

We should investigate if normal tsc --watch --incremental is now fast enough for the main VS Code build

/cc @rbuckton @weswigham

[mjbvz]

I believe that @rbuckton mentioned that TypeScript is using gulp + normal tsc. Relevant code: https://github.com/microsoft/TypeScript/blob/47182b543a2a0af7c71b9a098ceb4c419d745e2e/Gulpfile.js#L24

[weswigham]

Probably worth noting: our infra around our build is way more complex than your probably needs to be, since we needed to wrap tsc -b, and lift project dependency semantics from gulp dependencies into project reference dependencies. Y'all probably don't need any of that and could just exec("tsc --watch --incremental").

[jrieken]

• Bring tsconfig.json in a state that allows to compile via npx tsc -p src/tsconfig.json. That's not possible due to some d.ts conflicts
• Measure & compare... E.g. measure on different OSes, measure different changes, e.g change code inside a function, change the signature of a private property, change the signature of an export variable

[jrieken]

Memory-safe command to run the TypeScript compiler: node --max-old-space-size=4096 ./node_modules/.bin/tsc -p src/tsconfig.json

[jrieken]

@Tyriar There seems to an issue with xterm and its definition files. They appear two times causing these errors

node_modules/xterm-addon-search/typings/xterm-addon-search.d.ts(40,16): error TS2300: Duplicate identifier 'SearchAddon'.
node_modules/xterm-addon-web-links/typings/xterm-addon-web-links.d.ts(13,16): error TS2300: Duplicate identifier 'WebLinksAddon'.
node_modules/xterm/typings/xterm.d.ts(16,15): error TS2300: Duplicate identifier 'FontWeight'.
node_modules/xterm/typings/xterm.d.ts(21,15): error TS2300: Duplicate identifier 'LogLevel'.
node_modules/xterm/typings/xterm.d.ts(26,15): error TS2300: Duplicate identifier 'RendererType'.
node_modules/xterm/typings/xterm.d.ts(351,16): error TS2300: Duplicate identifier 'Terminal'.
src/typings/xterm-addon-search.d.ts(41,15): error TS2300: Duplicate identifier 'SearchAddon'.
src/typings/xterm-addon-web-links.d.ts(52,15): error TS2300: Duplicate identifier 'WebLinksAddon'.
src/typings/xterm.d.ts(16,14): error TS2300: Duplicate identifier 'FontWeight'.
src/typings/xterm.d.ts(21,14): error TS2300: Duplicate identifier 'LogLevel'.
src/typings/xterm.d.ts(26,14): error TS2300: Duplicate identifier 'RendererType'.
src/typings/xterm.d.ts(351,15): error TS2300: Duplicate identifier 'Terminal'.
src/typings/xterm.d.ts(1078,12): error TS2300: Duplicate identifier 'Terminal'.

Removing (or excluding) typings/xterms*.d.ts doesn't seem to be enough as the types seems to mismatch and having only node_modules/xterm/typings/xterm*.d.ts isn't enough.

Can you please take a look and fix. Thanks

[Tyriar]

Private xterm API access is now declared inside contrib/terminal/ instead of typings/, and typings/xterm* is excluded from src/tsconfig.json

[jrieken]

Thanks @Tyriar. We are now in a state where we can start measuring and comparing tsc vs gulp-tsb

[jrieken]

Some measurements that I have done, all happened in linkedList.ts. I ran through the steps below, always three times in a row and noted the median (make change, save, wait for compiler, undo change, save, wait, repeat)

1. line 52 => change local variable name newNode to neNode
2. line 51 => remove argument of private function , atTheEnd: boolean
3. line 51 => change name of private function _insert to _inser
4. line 47 => change name of argument element to elemnt
5. line 13 => change name of field next to nxt
6. line 43 => change name of unshift to unshft

	gulp-tsb	tsc -w
1	<1	<1
2	<1	<1
3	<1	30 ❗️
4	<1	30 ❗️
5	<1	<1
6	<1	30 ❗️

For completeness, commands and output files. tsb: yarn watch-client | tee ~/tsb.out.txt, tsc: node --max-old-space-size=4096 ./node_modules/.bin/tsc -p src/tsconfig.json -w | tee -a ~/tsc.out.txt

tsb.out.txt
tsc.out.txt
tsc2.out.txt

It seems that tsc -w is on par with "local" changes (scenarios 1, 2, and 5) but much, much slower with changes that "cause a ripple", e.g changing externally visible symbols seem to have a constant cost (full re-check running?)

@weswigham Can you explain why tsc is so slow here? I think tsc and tsb both use a similar strategy which is based on declaration-files/information. E.g when a change triggers a change in the (to be emitted) foo.d.ts file then a deep analysis is needed.

In tsb however, we use a reverse dependency graph of files and re-check only the neighbours and we don't recurse on a neighbour when its declaration-information didn't change. E.g. the unshift method from scenario 6 is only used in three places and it's used in "local" ways, e.g the d.ts-files of those consumers don't change and therefore we exit early.

[rbuckton]

@jrieken: Have you tried tsc with --incremental?

• http://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-4.html#faster-subsequent-builds-with-the---incremental-flag
• http://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-5.html#--incremental-improvements

[jrieken]

Using --incremental doesn't change things, something like scenario 6 is still ~30seconds

[jrieken]

closing as this won't happen anytime soon.

[Tyriar]

FYI created microsoft/TypeScript#33323 which I consider a blocker if we wanted to adopt project references (after we're use tsc -p)

[mjbvz]

I've opened microsoft/TypeScript#33329 for this. I know there's been a lot of recenty TS work on --incremental and want to make sure there is awareness of this issue