Skip to content

CI should cache submodules? #2387

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
smarter opened this issue May 6, 2017 · 3 comments
Closed

CI should cache submodules? #2387

smarter opened this issue May 6, 2017 · 3 comments
Assignees
@allanrenucci
Labels
area:infrastructure prio:low

Comments

@smarter
Copy link
Member

smarter commented May 6, 2017

The first thing the CI does on every build is git submodule update --init, this takes 23 seconds, most of that time is spent checking out lampepfl/scala twice (it's dumb that git doesn't realize it's doing the same thing twice here). It would be nice if we could cache this. It may also be worth merging scala-backend and scala2-library in a single submodule just to make it easier on everyone trying out dotty (on a bad connection a recursive clone might take a looong time).
@smarter
Copy link
Member Author

smarter commented Jan 11, 2018

@allanrenucci Do you think this is doable or should we close?

@allanrenucci
Copy link
Contributor

I'll will experiment with this over the next week. I think there are simple optimisations I can try such as cloning the submodules in parallel.

Regarding caching, I don't see an easy solution

@allanrenucci allanrenucci self-assigned this Jan 11, 2018
allanrenucci added a commit to allanrenucci/dotty that referenced this issue Jan 12, 2018
@allanrenucci
Mitigate scala#2387: Improve cloning logic
de26b52 
- Clone submodules in parallel
- Improve workaround to scala#3415. Don't rely on GitHub to perform the merge
  on PRs
allanrenucci added a commit to dotty-staging/dotty that referenced this issue Jan 15, 2018
@allanrenucci
Mitigate scala#2387: Improve cloning logic
13d2cd4 
- Clone submodules in parallel
- Improve workaround to scala#3415. Don't rely on GitHub to perform the merge
  on PRs
allanrenucci added a commit to allanrenucci/dotty that referenced this issue Jan 16, 2018
@allanrenucci
Mitigate scala#2387: Improve cloning logic
6d94b0f 
- Clone submodules in parallel
- Improve workaround to scala#3415. Don't rely on GitHub to perform the merge
  on PRs
allanrenucci added a commit to allanrenucci/dotty that referenced this issue Jan 16, 2018
@allanrenucci
Mitigate scala#2387: Improve cloning logic
dc8a2bd 
- Clone submodules in parallel
- Improve workaround to scala#3415. Don't rely on GitHub to perform the merge
  on PRs
allanrenucci added a commit to allanrenucci/dotty that referenced this issue Jan 16, 2018
@allanrenucci
Mitigate scala#2387: Improve cloning logic
30df7ac 
- Clone submodules in parallel
- Improve workaround to scala#3415. Don't rely on GitHub to perform the merge
  on PRs
allanrenucci added a commit that referenced this issue Jan 16, 2018
@allanrenucci
Merge pull request #3850 from allanrenucci/drone-clone
24057cb 
Mitigate #2387: Improve cloning logic
@allanrenucci
Copy link
Contributor

Closing as I don't plan to implement caching for submodules since I minimised the time spent cloning submodules by doing it in parallel. (ref/ #3850)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@allanrenucci allanrenucci

Labels
area:infrastructure prio:low
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@smarter @allanrenucci