Epic: Incremental Workspaces

[svenefftinge]

Starting prebuilds should be as fast as possible.

Incremental Workspaces would allow starting a workspace using a prebuild that ran for an older commit (based on commit history) and then run the init tasks on top of that when the workspace starts. This allows benefitting from the caching on the file system just like you would in a local dev environment.

Activating this behavior would mean that users never get to see the 'prebuild in flight' view anymore as we would ignore non-finished rebuilds and only consider finished ones when starting a workspace.

Disclaimer: I am aware that in some cases where the workspace is really large, loading prebuild data can take more time than the actual full build. So it would be desired to not use prebuilds at all.

internal RFC https://www.notion.so/gitpod/Incremental-Workspaces-08507a1d569c4c15ba6493d7380a5637
Related to #12583

---

EDIT by @jankeromnes -- steps:

• Implement a 'Use Last Successful Prebuild' workspace creation mode #13801
• When starting a workspace in this new mode, also make Gitpod re-run the prebuild tasks to incrementally update the build with the latest Git state (similar to what incremental prebuilds do) [content-service] incremental workspace init #14140
• Add a feature flag that ignores running prebuilds and automatically uses the best matching prebuild as base.
• Implement a Project Setting that makes this new workspace creation mode the default for a project, such that you never see the "Prebuild in Progress" waiting screen again

[jankeromnes]

Thanks @svenefftinge for proposing this and #12583! I agree that implementing these two ideas could be a game changer, allowing very large + high-frequency projects to benefit from prebuilds as well. (And I also really like the value / implementation cost ratios of these two issues. 💯)

We should allow starting workspaces in this mode as well, so that if I skip a prebuild I still get an almost initialized state and then see the incremental build happening in my interactive workspace, so I can already start browsing files or do other things (including canceling any builds, because I'm not interested).

On the "Prebuild in progress" UX, I wonder whether it's better to:

1. Continue offering the "Skip prebuild, start from scratch" option, while also offering a new alternative option "Skip prebuild, use last successful prebuild instead"

2. Or, completely remove the "from scratch" option and replace it with only the "use last successful prebuild"?

I.e. maybe sometimes, you do actually want to start from scratch? (E.g. if you've recently committed a change to your build system, and you know that any old prebuild will be heavy, slow, and 100% useless?)

[svenefftinge]

On the "Prebuild in progress" UX ...

I think this change together with #12583 would make it so that you don't see that UI anymore. We should still think about improving it maybe, but let's explore that separately and later.