Manage one checkout for all projects

[ericholscher]

There would be a sizable win if we changed from having a checkout for every version of a project to having one central checkout, that gets cloned and then deleted into each version on build. This would save lots of bandwidth and disk space, because we could keep a much larger cache of clones.

The main space issue is actually around the virtualenvs, but this would be a good start to addressing this.

[humitos]

We already implemented #4939 that forces git to download only the latest 50 commits. It's not exactly the same as what this issue proposes but it's related.

Considering that we are removing all the checkouts at the end of the day if not recently updated, I suppose that by having only one checkout for all the versions we are not going to gain too much space here.

[stsewd]

This would be useful if we save the one checkout in one server/location, right now we have several builders and it wouldn't make much difference if we save one checkout per builder.

[humitos]

We are talking about having volatile servers in the middle term because different reasons (disk costs, reproducibility, etc), so I think this issue is not valid anymore. I'm closing it here.