Improve engineering overhead of extensions

[joaomoreno]

Considering we want to keep the core extensions within the VS Code repo for the time being, let's look into improving the engineering overhead of that situation. We want to avoid recompiling extensions unnecessarily, something which happens often in the CI as well as in each dev box.

Here's what I'm thinking:

1. Each extension should have a few npm scripts: package (which creates a VSIX), watch (which compiles the sources in watch mode).
2. At any CI run, it should be possible to figure out what was the last commit which affected any given extension (something like git log -1 -- extension/NAME, or similar). We can use that commit ID as a cache key to save the extension's built VSIX in a storage account.
3. During CI, the build machine should check whether each extension's cache is fresh or stale. If stale, the extension should be packaged with yarn && yarn package and the resulting VSIX should be cached in the storage account. If fresh, the same Marketplace VSIX mechanism should be used to fetch the VSIX from a remote location, but this time from the storage account.
4. Additionally, we can improve the local yarn watch experience to fetch the VSIXs from the same storage account, which should have public read access. Given that a dev has no changes which affect an extension, the VSIX should just be used. As soon as changes appear locally, yarn watch should do the right thing automatically (yarn && yarn watch on the extension folder, and signal code-oss to use it instead).

This way we can reduce build times, dev times and keep having extensions in the microsoft/vscode repo.

[lszomoru]

Are we currently using TypeScript project references? Is this something that could help with the dev box build times?

[lszomoru]

For the CI runs, it would also be great if we could only run the tests, that test the code that has been modified in the commit.

[joaomoreno]

Are we currently using TypeScript project references? Is this something that could help with the dev box build times?

The interface between extensions and core isn't at the TypeScript level: it's at the VSIX level. Project references help with creating definition interfaces between projects, but we already craft ours manually: vscode.d.ts.

For the CI runs, it would also be great if we could only run the tests, that test the code that has been modified in the commit.

This is a whole different story, much harder to accomplish. But generally a good idea yes.

[joaomoreno]

After 2 full days of investment, I decided to put a pin on this. Work is in joao/extensions. This would need a considerable amount of time invested, most of it spent on endlessly running builds and trying to get them to succeed. My engineering passion is big, but I think it's been way too long to be solely focusing on this. I want to do feature work for now.

The main problems with this are:

1. Extension builds are intertwined with global dependencies and different build engines (webpack, gulp) and multiple outputs (desktop, web). It is not trivial to break this apart.
2. Developer experience is unknown, if we decide move to a complete webpack solution. Should we even move to esbuild?
3. Even if we get extension builds cached at the VSIX level, core integration tests still depend on having some extensions compiled apart from what they ship on their VSIXs.

cc @alexdima @eamodio @Tyriar

[TylerLeonhardt]

Talked to @joaomoreno about this and I think we're gonna put a pin in this. The engineering overhead isn't worth the effort at this time.