Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fill in gpuCI build script #1

Merged
merged 9 commits into from
Jul 30, 2021
10 changes: 10 additions & 0 deletions continuous_integration/gpuci/axis.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
PYTHON_VER:
- 3.8

CUDA_VER:
- 11.2

LINUX_VER:
- ubuntu18.04

excludes:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

64 changes: 63 additions & 1 deletion continuous_integration/gpuci/build.sh
Original file line number Diff line number Diff line change
@@ -1 +1,63 @@
# TODO
##############################################
# Dask GPU build and test script for CI #
##############################################
set -e
NUMARGS=$#
ARGS=$*

# Arg parsing function
function hasArg {
(( ${NUMARGS} != 0 )) && (echo " ${ARGS} " | grep -q " $1 ")
}

# Set path and build parallel level
export PATH=/opt/conda/bin:/usr/local/cuda/bin:$PATH
export PARALLEL_LEVEL=${PARALLEL_LEVEL:-4}

# Set home to the job's workspace
export HOME="$WORKSPACE"

# Switch to project root; also root of repo checkout
cd "$WORKSPACE"

# Determine CUDA release version
export CUDA_REL=${CUDA_VERSION%.*}

################################################################################
# SETUP - Check environment
################################################################################

gpuci_logger "Check environment variables"
env

gpuci_logger "Check GPU usage"
nvidia-smi

gpuci_logger "Activate conda env"
. /opt/conda/etc/profile.d/conda.sh
conda activate rapids

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change to conda activate dask


gpuci_logger "Install dask dependencies"
gpuci_conda_retry env update --name rapids --file "$WORKSPACE/continuous_integration/environment-3.8.yaml"

gpuci_logger "Install testing dependencies"
gpuci_mamba_retry install -y \
cudf=21.08 \
cupy \
cudatoolkit=$CUDA_REL

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually you might still need to install distributed manually.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think distributed should get picked up by the pip installation of Dask, but in any case it isn't being used by any of the GPU tests here so it can probably be skipped for now


gpuci_logger "Install dask"
python setup.py install

gpuci_logger "Check compiler versions"
python --version
$CC --version
$CXX --version

gpuci_logger "Check conda environment"
conda info
conda config --show-sources
conda list --show-channel-urls

gpuci_logger "Python py.test for dask"
py.test $WORKSPACE -n 4 -v -m gpu --junitxml="$WORKSPACE/junit-dask.xml" --cov-config="$WORKSPACE/.coveragerc" --cov=dask --cov-report=xml:"$WORKSPACE/dask-coverage.xml" --cov-report term