stable, dev PyTorch in Dockerfile and conda gh actions #3074

ydcjeff · 2020-08-20T13:06:26Z

What does this PR do?

Add Docker image installed with pytorch nightly, PyTorch & Conda gh action tested with pytorch nightly

Before submitting

Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests?
Did you verify new and existing tests pass locally with your changes?
If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

dockers/cuda-nightly/Dockerfile

codecov · 2020-08-20T14:43:58Z

Codecov Report

Merging #3074 into master will decrease coverage by 2%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #3074    +/-   ##
=======================================
- Coverage      88%     86%    -2%     
=======================================
  Files         109     109            
  Lines        8436    8043   -393     
=======================================
- Hits         7409    6916   -493     
- Misses       1027    1127   +100

dockers/cuda-nightly/Dockerfile

Borda

pls, update the cuda based image such that the 1.7/nightmli is just an argument so we avoid maintaining too many images...

ydcjeff · 2020-08-22T08:11:29Z

@Borda torch is not installed as locked version in build-cuda github action.
https://github.com/PyTorchLightning/pytorch-lightning/runs/1013619899?check_suite_focus=true
also, i think installing different torch versions with conda might be easier than installing with pip

Borda · 2020-08-24T07:32:17Z

maybe but in Conda env we are running only CPU tests compare to the CUDA-pip image we run in Drone with the full suite case

ydcjeff · 2020-08-24T14:56:54Z

Found out pytorch CUDA image itself is also using conda, so go with it right now.
would you like to change to pip?

Borda · 2020-08-24T15:37:45Z

Found out pytorch CUDA image itself is also using conda, so go with it right now.
would you like to change to pip?

Well but is conda activated by default?

ydcjeff · 2020-08-24T15:43:59Z

Found out pytorch CUDA image itself is also using conda, so go with it right now.
would you like to change to pip?

Well but is conda activated by default?

yes

ydcjeff · 2020-08-26T13:53:30Z

for GPU testing with stable and dev PyTorch on Drone, shall we do testing in parallel or one after another?
I go for one after another
WDYT?

ydcjeff · 2020-08-27T10:17:47Z

@Borda I have noticed that PyTorch & Conda github actions is installing gpu version of pytorch while we are not testing with gpu. Is this for some reasons or shall i add cpuonly package in environment.yml to save a few pkg installing minutes?

ydcjeff · 2020-09-03T13:26:26Z

@Borda mind review this so far?
so that i can proceed #3074 (comment) and #3074 (comment)

Borda · 2020-09-03T14:11:36Z

btw, is it still wip?

ydcjeff · 2020-09-03T14:18:26Z

btw, is it still wip?

@Borda I haven't worked on Drone CI for GPU testing, so can be said wip
waiting for review of current base-cuda Dokerfile, so that i can continue

ydcjeff · 2020-09-04T17:14:16Z

@Borda do I need to change this docker image?
https://github.com/PyTorchLightning/pytorch-lightning/blob/64ee25ed9cd1d44bb276b61683bbdfee0b00ff9d/.drone.yml#L23

Borda · 2020-09-04T17:26:11Z

@Borda do I need to change this docker image?
https://github.com/PyTorchLightning/pytorch-lightning/blob/64ee25ed9cd1d44bb276b61683bbdfee0b00ff9d/.drone.yml#L23

you cannot until it is build for the first time and push to the docker hub...

ydcjeff · 2020-09-04T17:29:34Z

@Borda do I need to change this docker image?
https://github.com/PyTorchLightning/pytorch-lightning/blob/64ee25ed9cd1d44bb276b61683bbdfee0b00ff9d/.drone.yml#L23

you cannot until it is build for the first time and push to the docker hub...

Ok then, shall we build the docker from this PR without merging into master?
OR build the docker from master and make another follow PR of this?

ydcjeff · 2020-09-04T18:17:32Z

@Borda ready to review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

Borda · 2020-09-17T13:38:00Z

@ydcjeff seems that tests for pt 1.7 are failing so I would recommend to keep the CI ready and just comment for now...
then make another PR which would fix the failing test and uncomment the testing

SkafteNicki

LGTM, great job

rohitgr7

All good. Nice work!

mergify bot requested a review from a team August 20, 2020 13:08

ydcjeff commented Aug 20, 2020

View reviewed changes

dockers/cuda-nightly/Dockerfile Outdated Show resolved Hide resolved

ydcjeff commented Aug 20, 2020

View reviewed changes

dockers/cuda-nightly/Dockerfile Outdated Show resolved Hide resolved

Borda added this to the 0.9.x milestone Aug 20, 2020

Borda reviewed Aug 21, 2020

View reviewed changes

dockers/cuda-nightly/Dockerfile Outdated Show resolved Hide resolved

mergify bot requested a review from a team August 21, 2020 17:17

Borda requested changes Aug 21, 2020

View reviewed changes

mergify bot requested a review from a team August 21, 2020 17:19

Borda added feature Is an improvement or enhancement ci Continuous Integration labels Aug 24, 2020

Borda self-assigned this Aug 27, 2020

ydcjeff requested review from Borda and removed request for a team September 2, 2020 14:49

mergify bot requested a review from a team September 2, 2020 14:49

ydcjeff changed the title ~~[WIP] build, test with stable and dev PyTorch~~ stable, dev PyTorch in Dockerfile and gh actions Sep 4, 2020

Jeff Yang and others added 12 commits September 17, 2020 14:58

Update .github/workflows/ci_pt-conda.yml

dc2e055

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

Update .github/workflows/ci_pt-conda.yml

01896b5

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

update

f6a931c

update

f0c13ac

fix cmd

dd6f907

filled &&

26f31ec

fix

bbd10a4

add -y

c12b02a

torchvision >0.7 allowed

8e324b3

explicitly install torchvision

6aacc45

use HOROVOD_GPU_OPERATIONS env variable

c2feb96

CI

87d3864

Borda requested a review from williamFalcon September 17, 2020 13:06

Borda added the ready PRs ready to be merged label Sep 17, 2020

Borda added 2 commits September 17, 2020 15:40

skip 1.7

dbaa358

table

70d5dd4

teddykoker approved these changes Sep 17, 2020

View reviewed changes

Borda requested review from awaelchli, jeremyjordan, nateraw, rohitgr7 and yukw777 September 17, 2020 16:08

SkafteNicki approved these changes Sep 17, 2020

View reviewed changes

rohitgr7 approved these changes Sep 17, 2020

View reviewed changes

Borda merged commit 8be79a9 into Lightning-AI:master Sep 17, 2020

Borda mentioned this pull request Sep 17, 2020

add support for 1.7 nightly #3531

Closed

2 tasks

ydcjeff deleted the nightly branch September 18, 2020 02:22

Borda mentioned this pull request Sep 18, 2020

build more docker configs #3533

Merged

1 task

ydcjeff mentioned this pull request Sep 24, 2020

Lightning docker image based on base-cuda #3637

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stable, dev PyTorch in Dockerfile and conda gh actions #3074

stable, dev PyTorch in Dockerfile and conda gh actions #3074

ydcjeff commented Aug 20, 2020 •

edited by Borda

Loading

codecov bot commented Aug 20, 2020 •

edited

Loading

Borda left a comment

ydcjeff commented Aug 22, 2020 •

edited

Loading

Borda commented Aug 24, 2020

ydcjeff commented Aug 24, 2020

Borda commented Aug 24, 2020

ydcjeff commented Aug 24, 2020

ydcjeff commented Aug 26, 2020

ydcjeff commented Aug 27, 2020

ydcjeff commented Sep 3, 2020

Borda commented Sep 3, 2020

ydcjeff commented Sep 3, 2020 •

edited

Loading

ydcjeff commented Sep 4, 2020

Borda commented Sep 4, 2020

ydcjeff commented Sep 4, 2020

ydcjeff commented Sep 4, 2020

Borda commented Sep 17, 2020

SkafteNicki left a comment

rohitgr7 left a comment

stable, dev PyTorch in Dockerfile and conda gh actions #3074

stable, dev PyTorch in Dockerfile and conda gh actions #3074

Conversation

ydcjeff commented Aug 20, 2020 • edited by Borda Loading

What does this PR do?

Before submitting

PR review

Did you have fun?

codecov bot commented Aug 20, 2020 • edited Loading

Codecov Report

Borda left a comment

Choose a reason for hiding this comment

ydcjeff commented Aug 22, 2020 • edited Loading

Borda commented Aug 24, 2020

ydcjeff commented Aug 24, 2020

Borda commented Aug 24, 2020

ydcjeff commented Aug 24, 2020

ydcjeff commented Aug 26, 2020

ydcjeff commented Aug 27, 2020

ydcjeff commented Sep 3, 2020

Borda commented Sep 3, 2020

ydcjeff commented Sep 3, 2020 • edited Loading

ydcjeff commented Sep 4, 2020

Borda commented Sep 4, 2020

ydcjeff commented Sep 4, 2020

ydcjeff commented Sep 4, 2020

Borda commented Sep 17, 2020

SkafteNicki left a comment

Choose a reason for hiding this comment

rohitgr7 left a comment

Choose a reason for hiding this comment

ydcjeff commented Aug 20, 2020 •

edited by Borda

Loading

codecov bot commented Aug 20, 2020 •

edited

Loading

ydcjeff commented Aug 22, 2020 •

edited

Loading

ydcjeff commented Sep 3, 2020 •

edited

Loading