Engineering: Improve build perf

[joaomoreno]

These measurements were made on my Linux beefy box.

Running yarn watch will spawn 3 node processes:

• yarn run watch, takes 700MB RAM
• node gulp watch --max-old_space_size=4095, takes 600MB RAM
• gulp watch --no-respawning, takes 4.5GB RAM (it first goes up to 5GB, then garbage collects)

The entire build in a dev box requires +6GB RAM.

On top of that, everything runs in the same one thread, even though we have a lot of potential parallelization (each extension's compilation could run independently) and we all have multi-core CPUs in our machines.

So, we're currently in a bad spot both in terms of CPU and memory. We need ideas on how to break this up and improve our situation.

@mjbvz Can TypeScript help us in anyway here? Is there such a feature which allows us to easily split our code base into components? I remember something was in the backlog, I'm just not aware of its status.

cc @microsoft/vscode

[jrieken]

I do remember that we run all gulp-tsb in the same process because we "coordinate" the printing of error messages and that would be harder if it happens in separate processes.

[Tyriar]

@jrieken looked at moving off of gulp-tsb in favor of tsc a while ago but I think there were performance issues that would slow us down too much, instead of taking lots of RAM it would take a much longer time to recompile under some cases. A possible solution to this would be to use project references everywhere but that's a huge change and also comes with several other issues like the editor acting up sometimes.

[mjbvz]

See microsoft/TypeScript#33329 for moving off of gulp tsb. That is the proper long term fix (there still may be memory usage concerns with tsc but it's better to be on standard tooling)