Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Support CUDA11.0 on CD #19295

Merged
merged 4 commits into from
Oct 6, 2020
Merged

Support CUDA11.0 on CD #19295

merged 4 commits into from
Oct 6, 2020

Conversation

waytrue17
Copy link
Contributor

Description

Enable CUDA 11.0 on CD nightly pipeline and release pipeline
Remove CUDA 9.2 and CUDA 10.0 from CD

@mxnet-bot
Copy link

Hey @waytrue17 , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [windows-gpu, unix-cpu, edge, unix-gpu, sanity, miscellaneous, centos-cpu, clang, website, windows-cpu, centos-gpu]

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@waytrue17
Copy link
Contributor Author

@leezu @szha can you please review, thanks.

leezu
leezu reviewed Oct 6, 2020
View reviewed changes
Copy link
Contributor

@leezu leezu left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

centos7_gpu_cu92 is still listed in Jenkins_steps.groovy for testing "oldest supported cuda". As Cuda 11 is now supported and we generally support the last 2 major cuda versions, you need to replace it with centos7_gpu_cu102.

leezu
leezu approved these changes Oct 6, 2020
View reviewed changes
Copy link
Contributor

@leezu leezu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @waytrue17

@leezu leezu merged commit 3042df0 into apache:master Oct 6, 2020
@waytrue17 waytrue17 mentioned this pull request Oct 7, 2020
Merged
2 tasks
@access2rohit access2rohit mentioned this pull request Feb 17, 2021
Closed
13 tasks
access2rohit pushed a commit to access2rohit/incubator-mxnet that referenced this pull request Feb 19, 2021
@waytrue17
Enable CUDA 11.0 on nightly development builds (apache#19295)
e7d5935 
Remove CUDA 9.2 and CUDA 10.0
@access2rohit access2rohit mentioned this pull request Feb 19, 2021
Merged
access2rohit pushed a commit to access2rohit/incubator-mxnet that referenced this pull request Feb 23, 2021
@waytrue17
Enable CUDA 11.0 on nightly development builds (apache#19295)
dc26b74 
Remove CUDA 9.2 and CUDA 10.0
Zha0q1 pushed a commit that referenced this pull request Feb 25, 2021
@access2rohit @waytrue17 @szha
[BACKPORT]Enable CUDA 11.0 on nightly + CUDA 11.2 on pip (#19295)(#19764
57dddd1 
) (#19930)

* Enable CUDA 11.0 on nightly development builds (#19295)

Remove CUDA 9.2 and CUDA 10.0

* [PIP] add build variant for cuda 11.2 (#19764)

* adding ci docker files for cu111 and cu112

* removing previous CUDA make versions and adding support for cuda11.2

Co-authored-by: waytrue17 <52505574+waytrue17@users.noreply.github.com>
Co-authored-by: Sheng Zha <szha@users.noreply.github.com>
Co-authored-by: Rohit Kumar Srivastava <srivastava.141@buckeyemail.osu.edu>
access2rohit added a commit to access2rohit/incubator-mxnet that referenced this pull request Mar 10, 2021
@access2rohit @waytrue17 @szha
[BACKPORT]Enable CUDA 11.0 on nightly + CUDA 11.2 on pip (apache#19295)(
f285a5f 
apache#19764) (apache#19930)

* Enable CUDA 11.0 on nightly development builds (apache#19295)

Remove CUDA 9.2 and CUDA 10.0

* [PIP] add build variant for cuda 11.2 (apache#19764)

* adding ci docker files for cu111 and cu112

* removing previous CUDA make versions and adding support for cuda11.2

Co-authored-by: waytrue17 <52505574+waytrue17@users.noreply.github.com>
Co-authored-by: Sheng Zha <szha@users.noreply.github.com>
Co-authored-by: Rohit Kumar Srivastava <srivastava.141@buckeyemail.osu.edu>
access2rohit added a commit to access2rohit/incubator-mxnet that referenced this pull request Mar 10, 2021
@access2rohit @waytrue17 @szha
[BACKPORT]Enable CUDA 11.0 on nightly + CUDA 11.2 on pip (apache#19295)(
264bd8a 
apache#19764) (apache#19930)

* Enable CUDA 11.0 on nightly development builds (apache#19295)

Remove CUDA 9.2 and CUDA 10.0

* [PIP] add build variant for cuda 11.2 (apache#19764)

* adding ci docker files for cu111 and cu112

* removing previous CUDA make versions and adding support for cuda11.2

Co-authored-by: waytrue17 <52505574+waytrue17@users.noreply.github.com>
Co-authored-by: Sheng Zha <szha@users.noreply.github.com>
Co-authored-by: Rohit Kumar Srivastava <srivastava.141@buckeyemail.osu.edu>
access2rohit added a commit to access2rohit/incubator-mxnet that referenced this pull request Mar 12, 2021
@access2rohit @waytrue17 @szha
[BACKPORT]Enable CUDA 11.0 on nightly + CUDA 11.2 on pip (apache#19295)(
408471a 
apache#19764) (apache#19930)

* Enable CUDA 11.0 on nightly development builds (apache#19295)

Remove CUDA 9.2 and CUDA 10.0

* [PIP] add build variant for cuda 11.2 (apache#19764)

* adding ci docker files for cu111 and cu112

* removing previous CUDA make versions and adding support for cuda11.2

Co-authored-by: waytrue17 <52505574+waytrue17@users.noreply.github.com>
Co-authored-by: Sheng Zha <szha@users.noreply.github.com>
Co-authored-by: Rohit Kumar Srivastava <srivastava.141@buckeyemail.osu.edu>
mseth10 added a commit that referenced this pull request Mar 14, 2021
@access2rohit @waytrue17
@szha @mseth10
[v1.8.x][BACKPORT]Stablizing CI and making binaries apache compliant (#…
a0535dd 
…20015)

* [BACKPORT]Enable CUDA 11.0 on nightly + CUDA 11.2 on pip (#19295)(#19764) (#19930)

* Enable CUDA 11.0 on nightly development builds (#19295)

Remove CUDA 9.2 and CUDA 10.0

* [PIP] add build variant for cuda 11.2 (#19764)

* adding ci docker files for cu111 and cu112

* removing previous CUDA make versions and adding support for cuda11.2

Co-authored-by: waytrue17 <52505574+waytrue17@users.noreply.github.com>
Co-authored-by: Sheng Zha <szha@users.noreply.github.com>
Co-authored-by: Rohit Kumar Srivastava <srivastava.141@buckeyemail.osu.edu>

* [FEATURE]Migrating all CD pipelines to Ninja build + fix cu112 CD pipeline (#19974)

* migrating cd builds to ninja + removing static links to nvidia libs and leagacy cuda versions

* installing NCCL manually for cuda11.2 container

* set MSHADOW_USE_CUDNN=1 in CMakelists of mshadow to build properly for CUDNN support

* adding coverage to cd requirements file to fix cu100, cu101 and cu102 tests

* updating cd_test containers to ubuntu 18

* adding cmake config for linux native and adding USE_KV_STORE in linux_cpu

* updating zmq builds to statically link to libmxnet.so

* updating toolchains for r, clang and llvm for ubuntu18. OpenBlas Static link for 'distribution' build type only. Fix caffe build to use openCV 3. Remove leagacy Clang 3.9 from CI

* fix versions for pip install in ubuntu_core_sh add new search path for cuDNN

* finxing cudnn link problem for CUDA<=11.0

* adding library paths for libjpegturbo and lapack to fix failing CI on ubuntu 18 images

* removing ASAN integration test from miscellaneous CI as its not required

* fix lapack path for gpu builds

* correctly installing libjpegturbo for ubuntu 18

* updating docker images of r,jekyll,julia etc test containers+ fix java version to 8

* installing libomp.so

* removing debug test as its not required. Code clean-up

* adding alternate URL source for MNIST dataset as original website is down

* skipping flaky tests issue tracked #20011

Co-authored-by: Rohit Kumar Srivastava <srivastava.141@buckeyemail.osu.edu>

* update cudnn from 7 to 8 for cu102 (#19506)

* update cudnn from 7 to 8 for cu102 (#19522)

* downloading MNIST dataset from alternate URL (#20014)

Co-authored-by: Rohit Kumar Srivastava <srivastava.141@buckeyemail.osu.edu>

* fixing CI issue with v1.8.x

* addressing review comments

Co-authored-by: waytrue17 <52505574+waytrue17@users.noreply.github.com>
Co-authored-by: Sheng Zha <szha@users.noreply.github.com>
Co-authored-by: Rohit Kumar Srivastava <srivastava.141@buckeyemail.osu.edu>
Co-authored-by: Manu Seth <22492939+mseth10@users.noreply.github.com>
mseth10 added a commit to mseth10/incubator-mxnet that referenced this pull request Mar 15, 2021
@access2rohit @waytrue17
@szha @mseth10
[v1.8.x][BACKPORT]Stablizing CI and making binaries apache compliant (a…
4c2dbc0 
…pache#20015)

* [BACKPORT]Enable CUDA 11.0 on nightly + CUDA 11.2 on pip (apache#19295)(apache#19764) (apache#19930)

* Enable CUDA 11.0 on nightly development builds (apache#19295)

Remove CUDA 9.2 and CUDA 10.0

* [PIP] add build variant for cuda 11.2 (apache#19764)

* adding ci docker files for cu111 and cu112

* removing previous CUDA make versions and adding support for cuda11.2

Co-authored-by: waytrue17 <52505574+waytrue17@users.noreply.github.com>
Co-authored-by: Sheng Zha <szha@users.noreply.github.com>
Co-authored-by: Rohit Kumar Srivastava <srivastava.141@buckeyemail.osu.edu>

* [FEATURE]Migrating all CD pipelines to Ninja build + fix cu112 CD pipeline (apache#19974)

* migrating cd builds to ninja + removing static links to nvidia libs and leagacy cuda versions

* installing NCCL manually for cuda11.2 container

* set MSHADOW_USE_CUDNN=1 in CMakelists of mshadow to build properly for CUDNN support

* adding coverage to cd requirements file to fix cu100, cu101 and cu102 tests

* updating cd_test containers to ubuntu 18

* adding cmake config for linux native and adding USE_KV_STORE in linux_cpu

* updating zmq builds to statically link to libmxnet.so

* updating toolchains for r, clang and llvm for ubuntu18. OpenBlas Static link for 'distribution' build type only. Fix caffe build to use openCV 3. Remove leagacy Clang 3.9 from CI

* fix versions for pip install in ubuntu_core_sh add new search path for cuDNN

* finxing cudnn link problem for CUDA<=11.0

* adding library paths for libjpegturbo and lapack to fix failing CI on ubuntu 18 images

* removing ASAN integration test from miscellaneous CI as its not required

* fix lapack path for gpu builds

* correctly installing libjpegturbo for ubuntu 18

* updating docker images of r,jekyll,julia etc test containers+ fix java version to 8

* installing libomp.so

* removing debug test as its not required. Code clean-up

* adding alternate URL source for MNIST dataset as original website is down

* skipping flaky tests issue tracked apache#20011

Co-authored-by: Rohit Kumar Srivastava <srivastava.141@buckeyemail.osu.edu>

* update cudnn from 7 to 8 for cu102 (apache#19506)

* update cudnn from 7 to 8 for cu102 (apache#19522)

* downloading MNIST dataset from alternate URL (apache#20014)

Co-authored-by: Rohit Kumar Srivastava <srivastava.141@buckeyemail.osu.edu>

* fixing CI issue with v1.8.x

* addressing review comments

Co-authored-by: waytrue17 <52505574+waytrue17@users.noreply.github.com>
Co-authored-by: Sheng Zha <szha@users.noreply.github.com>
Co-authored-by: Rohit Kumar Srivastava <srivastava.141@buckeyemail.osu.edu>
Co-authored-by: Manu Seth <22492939+mseth10@users.noreply.github.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@leezu leezu leezu approved these changes

@aaronmarkham aaronmarkham Awaiting requested review from aaronmarkham aaronmarkham is a code owner

@marcoabreu marcoabreu Awaiting requested review from marcoabreu marcoabreu is a code owner

@szha szha Awaiting requested review from szha szha is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@waytrue17 @mxnet-bot @leezu