Cache docker builds #3659

ydcjeff · 2020-09-25T15:59:33Z

What does this PR do?

Used docker/build-push-action@v2 to able to cache

Before submitting

Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests?
Did you verify new and existing tests pass locally with your changes?
If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

codecov · 2020-09-25T17:06:11Z

Codecov Report

Merging #3659 into master will increase coverage by 3%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #3659    +/-   ##
=======================================
+ Coverage      90%     93%    +3%     
=======================================
  Files         111     111            
  Lines        8011    8011            
=======================================
+ Hits         7217    7448   +231     
+ Misses        794     563   -231

ydcjeff · 2020-09-30T12:14:00Z

@Borda I used docker/build-push-action@v2 to cache docker builds (can cache locally or on registry).
And we only cache base-cuda and base-xla builds in nightly events, since
caching in base-conda took more time than no cache, so no cache right now.

Also, storing cache locally took more time when restoring the cache.
I have tested in the forked repo and restoring local cache took ~3 mins while
restoring from registry cache took ~2s, idk why.

So, does pushing cache images (separate images) to docker hub sound good to you?

mergify · 2020-09-30T12:37:33Z

This pull request is now in conflict... :(

mergify · 2020-09-30T12:42:03Z

This pull request is now in conflict... :(

mergify · 2020-10-02T09:27:52Z

This pull request is now in conflict... :(

.github/workflows/nightly.yml

ydcjeff · 2020-10-02T10:34:09Z

@Borda Shall we install nvidia-dali in this PR or in a later PR?

Borda · 2020-10-02T11:21:51Z

@Borda Shall we install nvidia-dali in this PR or in a later PR?

I would keep it paired with the adding DALI example...

edenlightning · 2020-10-19T16:26:11Z

hi @ydcjeff mind taking a look at failing tests?

Borda

lgtm, just wait till all CI passes

.github/workflows/ci_dockers.yml

ydcjeff · 2020-10-20T08:02:50Z

@Borda Before merging, just want to make sure

this PR stores the cache metadata of the docker builds on docker hub with the separate names.
I found caching on docker hub is faster than caching in CI
Does it sounds good to you?

Borda · 2020-10-20T10:49:41Z

this PR stores the cache metadata of the docker builds on docker hub with the separate names.

how does it appear on Dockerhub, as a new entry/tag?

ydcjeff · 2020-10-20T11:12:43Z

this PR stores the cache metadata of the docker builds on docker hub with the separate names.

how does it appear on Dockerhub, as a new entry/tag?

It will appear as a new tag like base-cuda-cache-py3.8-torch1.6 (for eg), DIGEST would be UNKNOWN, OS/ARCH will be none, there will be no compressed size.

You can see here for example: https://hub.docker.com/r/ydcjeff/pl/tags (this is just for testing this PR)

.github/workflows/nightly.yml

tchaton

Great addition ! I didn't know about cache-from and cache-to. Really lean API from docker/build-push-action@v2 :) You have some small typos in the comment too.

.github/workflows/ci_dockers.yml

author ydcjeff <ydcjeff@outlook.com> 1601049378 +0630 committer ydcjeff <ydcjeff@outlook.com> 1601469495 +0630 cache docker builds lock horovod at 0.19.5 done [ci skip] [CI SKIP] use --cache-from [ci skip] typo and horovod [ci skip] exclude pt 1.3 py3.8 [ci skip] conda no cache [ci skip] fix

ydcjeff · 2020-10-25T12:10:59Z

I know 'Ready to go' is not meant to be added by PR author. But fix build-args commit changes small, so I think it's good to go.

ydcjeff marked this pull request as draft September 25, 2020 15:59

ydcjeff changed the title ~~Cache docker builds~~ Cache docker builds [CI SKIP] Sep 25, 2020

ydcjeff changed the title ~~Cache docker builds [CI SKIP]~~ Cache docker builds Sep 30, 2020

ydcjeff changed the title ~~Cache docker builds~~ [blocked by #3739]Cache docker builds Sep 30, 2020

ydcjeff marked this pull request as ready for review September 30, 2020 12:14

mergify bot requested a review from a team September 30, 2020 12:14

ydcjeff changed the title ~~[blocked by #3739]Cache docker builds~~ Cache docker builds Sep 30, 2020

ydcjeff mentioned this pull request Oct 2, 2020

Add Dali MNIST example #3721

Merged

7 tasks

Borda added the ci Continuous Integration label Oct 2, 2020

Borda reviewed Oct 2, 2020

View reviewed changes

.github/workflows/nightly.yml Outdated Show resolved Hide resolved

mergify bot requested a review from a team October 2, 2020 10:14

ydcjeff changed the title ~~Cache docker builds~~ Cache docker builds [CI SKIP] Oct 2, 2020

ydcjeff changed the title ~~Cache docker builds [CI SKIP]~~ Cache docker builds Oct 2, 2020

ydcjeff requested review from Borda and removed request for a team October 2, 2020 15:22

mergify bot requested a review from a team October 2, 2020 15:22

Borda added the feature Is an improvement or enhancement label Oct 19, 2020

Borda approved these changes Oct 19, 2020

View reviewed changes

.github/workflows/ci_dockers.yml Show resolved Hide resolved

mergify bot requested a review from a team October 19, 2020 21:40

Borda added this to the 1.0.x milestone Oct 20, 2020

ydcjeff commented Oct 20, 2020

View reviewed changes

.github/workflows/nightly.yml Show resolved Hide resolved

edenlightning modified the milestones: 1.0.3, 1.1 Oct 20, 2020

tchaton approved these changes Oct 20, 2020

View reviewed changes

.github/workflows/ci_dockers.yml Show resolved Hide resolved

awaelchli approved these changes Oct 25, 2020

View reviewed changes

ydcjeff added 6 commits October 25, 2020 10:40

revert

a83aea5

align with master [ci skip]

412b454

retry

90e0746

remove empty continuation lines

180d85f

add comment

7b66d87

ydcjeff requested review from ananyahjha93, justusschock, nateraw, SeanNaren, teddykoker and williamFalcon as code owners October 25, 2020 09:40

Borda added the ready PRs ready to be merged label Oct 25, 2020

Borda modified the milestones: 1.1, 1.0.x Oct 25, 2020

Merge remote-tracking branch 'upstream/master' into docker/cache

d20ba0b

ydcjeff removed the ready PRs ready to be merged label Oct 25, 2020

ydcjeff added 2 commits October 25, 2020 16:49

fix build-args

325e334

Merge branch 'master' into docker/cache

ec77dc0

ydcjeff added the ready PRs ready to be merged label Oct 25, 2020

ydcjeff merged commit d83c4e4 into Lightning-AI:master Oct 25, 2020

ydcjeff deleted the docker/cache branch October 25, 2020 12:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cache docker builds #3659

Cache docker builds #3659

ydcjeff commented Sep 25, 2020 •

edited

Loading

codecov bot commented Sep 25, 2020 •

edited

Loading

ydcjeff commented Sep 30, 2020

mergify bot commented Sep 30, 2020

mergify bot commented Sep 30, 2020

mergify bot commented Oct 2, 2020

ydcjeff commented Oct 2, 2020

Borda commented Oct 2, 2020

edenlightning commented Oct 19, 2020

Borda left a comment

ydcjeff commented Oct 20, 2020

Borda commented Oct 20, 2020

ydcjeff commented Oct 20, 2020 •

edited

Loading

tchaton left a comment •

edited

Loading

ydcjeff commented Oct 25, 2020

Cache docker builds #3659

Cache docker builds #3659

Conversation

ydcjeff commented Sep 25, 2020 • edited Loading

What does this PR do?

Before submitting

PR review

Did you have fun?

codecov bot commented Sep 25, 2020 • edited Loading

Codecov Report

ydcjeff commented Sep 30, 2020

mergify bot commented Sep 30, 2020

mergify bot commented Sep 30, 2020

mergify bot commented Oct 2, 2020

ydcjeff commented Oct 2, 2020

Borda commented Oct 2, 2020

edenlightning commented Oct 19, 2020

Borda left a comment

Choose a reason for hiding this comment

ydcjeff commented Oct 20, 2020

Borda commented Oct 20, 2020

ydcjeff commented Oct 20, 2020 • edited Loading

tchaton left a comment • edited Loading

Choose a reason for hiding this comment

ydcjeff commented Oct 25, 2020

ydcjeff commented Sep 25, 2020 •

edited

Loading

codecov bot commented Sep 25, 2020 •

edited

Loading

ydcjeff commented Oct 20, 2020 •

edited

Loading

tchaton left a comment •

edited

Loading