Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unpin CUDA docker image for GPU CI #12373

Merged
merged 117 commits into from
May 6, 2022
Merged

Unpin CUDA docker image for GPU CI #12373

merged 117 commits into from
May 6, 2022

Conversation

Borda
Copy link
Member

@Borda Borda commented Mar 18, 2022 •
edited by akihironitta
Loading

What does this PR do?

Does your PR introduce any breaking changes? If yes, please list them.

None

Before submitting

  • Was this discussed/approved via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

cc @Borda @tchaton @rohitgr7 @akihironitta @carmocca

@Borda Borda added the ci Continuous Integration label Mar 18, 2022
@Borda Borda requested review from tchaton and carmocca as code owners March 18, 2022 15:33
@Borda Borda requested review from akihironitta, awaelchli and a team March 18, 2022 15:34
akihironitta
akihironitta reviewed Mar 18, 2022
View reviewed changes
.azure-pipelines/gpu-benchmark.yml Outdated Show resolved Hide resolved
.azure-pipelines/gpu-tests.yml Outdated Show resolved Hide resolved
@Borda
Copy link
Member Author

Borda commented Mar 21, 2022

test_pruning_callback_ddp_spawn is failing
test_quantization[True-True-histogram] crashes with 344 Segmentation fault

@mergify mergify bot added the ready PRs ready to be merged label Mar 21, 2022
@carmocca carmocca added this to the 1.6 milestone Mar 21, 2022
@mergify mergify bot requested a review from a team March 21, 2022 17:13
@carmocca carmocca added the priority: 0 High priority task label Mar 21, 2022
tchaton
tchaton approved these changes Mar 22, 2022
View reviewed changes
Copy link
Contributor

@tchaton tchaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !

@Borda Borda requested a review from akihironitta March 22, 2022 14:32
@akihironitta akihironitta self-assigned this Mar 25, 2022
@akihironitta akihironitta marked this pull request as draft March 25, 2022 13:12
@akihironitta
Copy link
Contributor

Seeing some issues with horovod tests, and here're solutions just for the note:

If you see

open MPI error: A system-required executable either could not be found or was not executable by this user

try apt install openmpi-bin (openai/baselines#573)

If you see

The value of the MCA parameter "plm_rsh_agent" was set to a path
that could not be found:

  plm_rsh_agent: ssh : rsh

Please either unset the parameter, or check that the path is correct

try apt install ssh (open-mpi/ompi#3625)

@akihironitta akihironitta changed the title [wip] Unpin CUDA docker image for GPU CI Unpin CUDA docker image for GPU CI May 5, 2022
@akihironitta akihironitta marked this pull request as ready for review May 5, 2022 09:42
@akihironitta akihironitta requested a review from carmocca May 5, 2022 09:48
@akihironitta
Copy link
Contributor

@Borda @carmocca Would you mind having another look...?

Borda
Borda commented May 5, 2022
View reviewed changes
Copy link
Member Author

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool, THANK YOU @akihironitta !

.azure-pipelines/gpu-benchmark.yml Show resolved Hide resolved
dockers/base-cuda/Dockerfile Outdated Show resolved Hide resolved
@Borda Borda mentioned this pull request May 5, 2022
Merged
12 tasks
@Borda Borda enabled auto-merge (squash) May 5, 2022 15:38
@Borda Borda merged commit 7ce948e into master May 6, 2022
@Borda Borda deleted the ci/gpu branch May 6, 2022 02:56
@akihironitta
Copy link
Contributor

Looks strange to me that this PR got merged without three approvals...

@Borda
Copy link
Member Author

Borda commented May 6, 2022

Looks strange to me that this PR got merged without three approvals...

it had them in past, the re-request review is not the same as dismiss already got approvals...

carmocca pushed a commit that referenced this pull request May 9, 2022
@Borda @akihironitta
@pre-commit-ci @carmocca
Unpin CUDA docker image for GPU CI (#12373)
3394af2 
* unpin CUDA docker image for GPU CI
* Apply suggestions from code review

Co-authored-by: Aki Nitta <nitta@akihironitta.com>
Co-authored-by: Akihiro Nitta <akihiro@pytorchlightning.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
carmocca pushed a commit that referenced this pull request May 31, 2022
@Borda @akihironitta
@pre-commit-ci @carmocca
Unpin CUDA docker image for GPU CI (#12373)
26be3c9 
* unpin CUDA docker image for GPU CI
* Apply suggestions from code review

Co-authored-by: Aki Nitta <nitta@akihironitta.com>
Co-authored-by: Akihiro Nitta <akihiro@pytorchlightning.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
lexierule pushed a commit that referenced this pull request Jun 1, 2022
@Borda @akihironitta
@pre-commit-ci @lexierule
Unpin CUDA docker image for GPU CI (#12373)
c123173 
* unpin CUDA docker image for GPU CI
* Apply suggestions from code review

Co-authored-by: Aki Nitta <nitta@akihironitta.com>
Co-authored-by: Akihiro Nitta <akihiro@pytorchlightning.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@rohitgr7 rohitgr7 mentioned this pull request Jul 1, 2022
Merged
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@akihironitta akihironitta akihironitta approved these changes

@awaelchli awaelchli Awaiting requested review from awaelchli

@williamFalcon williamFalcon Awaiting requested review from williamFalcon

@SeanNaren SeanNaren Awaiting requested review from SeanNaren

@kaushikb11 kaushikb11 Awaiting requested review from kaushikb11

@rohitgr7 rohitgr7 Awaiting requested review from rohitgr7

@carmocca carmocca Awaiting requested review from carmocca

@tchaton tchaton Awaiting requested review from tchaton tchaton is a code owner

@justusschock justusschock Awaiting requested review from justusschock justusschock is a code owner

Assignees

@Borda Borda

@akihironitta akihironitta

Labels
ci Continuous Integration priority: 0 High priority task ready PRs ready to be merged
Projects
No open projects
Frameworks Planning
Status: Done
Milestone
pl:1.6.x
Development

Successfully merging this pull request may close these issues.
5 participants
@Borda @akihironitta @carmocca @tchaton @justusschock