Split cuda image into runtime and devel

Fixes: #12 Closes: #14
NVIDIA · Nov 21, 2015 · fe36ee3 · fe36ee3
1 parent 89f32ae
commit fe36ee3
Show file tree

Hide file tree

Showing 34 changed files with 277 additions and 48 deletions.
diff --git a/README.md b/README.md
@@ -10,32 +10,58 @@ Please be aware that this project is currently **experimental**.
 * Reproducible builds
 * Ease of deployment
 * Isolation of individual devices
-* Run across heterogeneous driver/toolkit environments *(e.g. CUDA 7.0 and 7.5 on a single host machine)*
+* Run across heterogeneous driver/toolkit environments
 * Requires only the NVIDIA driver
 
 ### Building images
 
 Images can be built on any machine running Docker, it doesn't require a NVIDIA GPU nor any driver installation.
 
-Building images is done through the Docker CLI or using the ```nvidia-docker``` wrapper similarly:
+#### CUDA
+Each CUDA image comes in two flavors:
+* ```runtime```: a lightweight image containing the bare minimum to deploy a pre-built application which uses CUDA.
+* ```devel```: extends the runtime image by adding the compiler toolchain, the debugging tools and the development files for the standard CUDA libraries. Use this image to compile a CUDA application from source.
+
+The ```devel``` image must be built **after** the ```runtime``` image:
 ```sh
-# With latest versions
-$ docker build -t cuda ubuntu/cuda/latest
+# Building a CUDA 7.5 development image based on Ubuntu
+docker build -t cuda:7.5-runtime ubuntu/cuda/7.5/runtime
+docker build -t cuda:7.5-devel ubuntu/cuda/7.5/devel
+docker tag cuda:7.5-devel cuda
 ```
 
 ```sh
-# With specific versions
-$ docker build -t cuda:7.5 ubuntu-14.04/cuda/7.5
+# Building a CUDA 7.5 development image based on CentOS
+docker build -t cuda:7.5-runtime centos/cuda/7.5/runtime
+docker build -t cuda:7.5-devel centos/cuda/7.5/devel
+docker tag cuda:7.5-devel cuda
 ```
 
 Alternatively, one can build an image directly from this repository:
 ```sh
-# With latest versions
-$ docker build -t cuda github.com/NVIDIA/nvidia-docker#:ubuntu/cuda/latest
+# Building a CUDA 7.5 development image based on Ubuntu
+docker build -t cuda:7.5-runtime github.com/NVIDIA/nvidia-docker#:ubuntu/cuda/7.5/runtime
+docker build -t cuda:7.5-devel github.com/NVIDIA/nvidia-docker#:ubuntu/cuda/7.5/devel
+docker tag cuda:7.5-devel cuda
+```
+
+#### cuDNN
+
+Currently, only cuDNN v2 based on CUDA 7.0 is supported, this image also comes in two flavors: ```cudnn-runtime``` and ```cudnn-devel```.
+```sh
+# Building a cuDNN image based on CUDA 7.0 runtime
+docker build -t cuda:7.0-runtime ubuntu/cuda/7.0/runtime
+docker build -t cuda:7.0-cudnn2-runtime ubuntu/cuda/7.0/runtime/cudnn2
+docker tag cuda:7.0-cudnn2-runtime cuda:cudnn-runtime
 ```
+
+Building the development image requires the ``cuda:7.0-devel`` image:
 ```sh
-# With specific versions
-$ docker build -t cuda:7.5 github.com/NVIDIA/nvidia-docker#:ubuntu-14.04/cuda/7.5
+# Building a cuDNN image based on CUDA 7.0 development
+docker build -t cuda:7.0-runtime ubuntu/cuda/7.0/runtime
+docker build -t cuda:7.0-devel ubuntu/cuda/7.0/devel
+docker build -t cuda:7.0-cudnn2-devel ubuntu/cuda/7.0/devel/cudnn2
+docker tag cuda:7.0-cudnn2-devel cuda:cudnn-devel
 ```
 
 ### NVIDIA Docker wrapper
@@ -46,7 +72,7 @@ GPUs are exported through a list of comma-separated IDs using the environment va
 The numbering is the same as reported by ```nvidia-smi``` or when running CUDA code with ```CUDA_DEVICE_ORDER=PCI_BUS_ID```, it is however **different** from the default CUDA ordering.
 
 ```sh
-$ GPU=0,1 ./nvidia-docker <docker-options> <docker-command> <docker-args>
+GPU=0,1 ./nvidia-docker <docker-options> <docker-command> <docker-args>
 ```
 
 ### CUDA requirements
@@ -66,10 +92,10 @@ CUDA toolkit version   | Minimum driver version  | Minimum GPU architecture
 ### Samples
 
 Once you have built the required images, a few examples are provided in the folder ```samples```.  
-The following assumes that you have an image in your repository named ```cuda``` (see ```samples/deviceQuery/Dockerfile```):
+The following assumes that you have an Ubuntu-based CUDA image in your repository (see ```samples/ubuntu/deviceQuery/Dockerfile```):
 ```sh
 # Run deviceQuery with one selected GPU
-$ docker build -t device_query samples/deviceQuery
+$ docker build -t device_query samples/ubuntu/deviceQuery
 $ GPU=0 ./nvidia-docker run device_query
 
 [ NVIDIA ] =INFO= Driver version: 352.39

diff --git a/centos-7/cuda/7.0/devel/Dockerfile b/centos-7/cuda/7.0/devel/Dockerfile
@@ -0,0 +1,17 @@
+FROM cuda:7.0-runtime
+MAINTAINER NVIDIA CORPORATION <digits@nvidia.com>
+
+RUN yum install -y \
+        cuda-core-$CUDA_PKG_VERSION \
+        cuda-misc-headers-$CUDA_PKG_VERSION \
+        cuda-command-line-tools-$CUDA_PKG_VERSION \
+        cuda-license-$CUDA_PKG_VERSION \
+        cuda-cublas-dev-$CUDA_PKG_VERSION \
+        cuda-cufft-dev-$CUDA_PKG_VERSION \
+        cuda-curand-dev-$CUDA_PKG_VERSION \
+        cuda-cusparse-dev-$CUDA_PKG_VERSION \
+        cuda-npp-dev-$CUDA_PKG_VERSION \
+        cuda-cudart-dev-$CUDA_PKG_VERSION \
+        cuda-driver-dev-$CUDA_PKG_VERSION \
+        gcc-c++ && \
+    rm -rf /var/cache/yum/*
diff --git a/centos-7/cuda/7.0/Dockerfile → centos-7/cuda/7.0/runtime/Dockerfile b/centos-7/cuda/7.0/Dockerfile → centos-7/cuda/7.0/runtime/Dockerfile
@@ -11,9 +11,17 @@ COPY cuda.repo /etc/yum.repos.d/cuda.repo
 ENV CUDA_VERSION 7.0
 LABEL com.nvidia.cuda.version="7.0"
 
+ENV CUDA_PKG_VERSION 7-0-7.0-28
 RUN yum install -y \
-        cuda-toolkit-7-0-7.0-28 \
-        gcc-c++ && \
+        cuda-nvrtc-$CUDA_PKG_VERSION \
+        cuda-cusolver-$CUDA_PKG_VERSION \
+        cuda-cublas-$CUDA_PKG_VERSION \
+        cuda-cufft-$CUDA_PKG_VERSION \
+        cuda-curand-$CUDA_PKG_VERSION \
+        cuda-cusparse-$CUDA_PKG_VERSION \
+        cuda-npp-$CUDA_PKG_VERSION \
+        cuda-cudart-$CUDA_PKG_VERSION && \
+    ln -s cuda-$CUDA_VERSION /usr/local/cuda && \
     rm -rf /var/cache/yum/*
 
 RUN echo "/usr/local/cuda/lib" >> /etc/ld.so.conf.d/cuda.conf && \

diff --git a/centos-7/cuda/7.0/cuda.repo → centos-7/cuda/7.0/runtime/cuda.repo b/centos-7/cuda/7.0/cuda.repo → centos-7/cuda/7.0/runtime/cuda.repo
diff --git a/centos-7/cuda/7.5/devel/Dockerfile b/centos-7/cuda/7.5/devel/Dockerfile
@@ -0,0 +1,17 @@
+FROM cuda:7.5-runtime
+MAINTAINER NVIDIA CORPORATION <digits@nvidia.com>
+
+RUN yum install -y \
+        cuda-core-$CUDA_PKG_VERSION \
+        cuda-misc-headers-$CUDA_PKG_VERSION \
+        cuda-command-line-tools-$CUDA_PKG_VERSION \
+        cuda-license-$CUDA_PKG_VERSION \
+        cuda-cublas-dev-$CUDA_PKG_VERSION \
+        cuda-cufft-dev-$CUDA_PKG_VERSION \
+        cuda-curand-dev-$CUDA_PKG_VERSION \
+        cuda-cusparse-dev-$CUDA_PKG_VERSION \
+        cuda-npp-dev-$CUDA_PKG_VERSION \
+        cuda-cudart-dev-$CUDA_PKG_VERSION \
+        cuda-driver-dev-$CUDA_PKG_VERSION \
+        gcc-c++ && \
+    rm -rf /var/cache/yum/*
diff --git a/centos-7/cuda/7.5/Dockerfile → centos-7/cuda/7.5/runtime/Dockerfile b/centos-7/cuda/7.5/Dockerfile → centos-7/cuda/7.5/runtime/Dockerfile
@@ -11,8 +11,17 @@ COPY cuda.repo /etc/yum.repos.d/cuda.repo
 ENV CUDA_VERSION 7.5
 LABEL com.nvidia.cuda.version="7.5"
 
+ENV CUDA_PKG_VERSION 7-5-7.5-18
 RUN yum install -y \
-        cuda-toolkit-7-5-7.5-18 && \
+        cuda-nvrtc-$CUDA_PKG_VERSION \
+        cuda-cusolver-$CUDA_PKG_VERSION \
+        cuda-cublas-$CUDA_PKG_VERSION \
+        cuda-cufft-$CUDA_PKG_VERSION \
+        cuda-curand-$CUDA_PKG_VERSION \
+        cuda-cusparse-$CUDA_PKG_VERSION \
+        cuda-npp-$CUDA_PKG_VERSION \
+        cuda-cudart-$CUDA_PKG_VERSION && \
+    ln -s cuda-$CUDA_VERSION /usr/local/cuda && \
     rm -rf /var/cache/yum/*
 
 RUN echo "/usr/local/cuda/lib" >> /etc/ld.so.conf.d/cuda.conf && \

diff --git a/centos-7/cuda/7.5/cuda.repo → centos-7/cuda/7.5/runtime/cuda.repo b/centos-7/cuda/7.5/cuda.repo → centos-7/cuda/7.5/runtime/cuda.repo
diff --git a/centos-7/cuda/latest b/centos-7/cuda/latest
diff --git a/samples/bandwidthTest/Dockerfile b/samples/bandwidthTest/Dockerfile
diff --git a/samples/centos/bandwidthTest/Dockerfile b/samples/centos/bandwidthTest/Dockerfile
@@ -0,0 +1,10 @@
+FROM cuda
+
+RUN yum install -y \
+        cuda-samples-$CUDA_PKG_VERSION && \
+    rm -rf /var/cache/yum/*
+
+WORKDIR /usr/local/cuda/samples/1_Utilities/bandwidthTest
+RUN make
+
+CMD ./bandwidthTest --mode=shmoo
diff --git a/samples/centos/deviceQuery/Dockerfile b/samples/centos/deviceQuery/Dockerfile
@@ -0,0 +1,10 @@
+FROM cuda
+
+RUN yum install -y \
+        cuda-samples-$CUDA_PKG_VERSION && \
+    rm -rf /var/cache/yum/*
+
+WORKDIR /usr/local/cuda/samples/1_Utilities/deviceQuery
+RUN make
+
+CMD ./deviceQuery
diff --git a/samples/centos/matrixMulCUBLAS/Dockerfile b/samples/centos/matrixMulCUBLAS/Dockerfile
@@ -0,0 +1,10 @@
+FROM cuda
+
+RUN yum install -y \
+        cuda-samples-$CUDA_PKG_VERSION && \
+    rm -rf /var/cache/yum/*
+
+WORKDIR /usr/local/cuda/samples/0_Simple/matrixMulCUBLAS
+RUN make
+
+CMD ./matrixMulCUBLAS -sizemult=10
diff --git a/samples/nvidia-smi/Dockerfile → samples/centos/nvidia-smi/Dockerfile b/samples/nvidia-smi/Dockerfile → samples/centos/nvidia-smi/Dockerfile
@@ -1,3 +1,3 @@
-FROM cuda:latest
+FROM cuda
 
 CMD nvidia-smi -q
diff --git a/samples/centos/vectorAdd/Dockerfile b/samples/centos/vectorAdd/Dockerfile
@@ -0,0 +1,10 @@
+FROM cuda
+
+RUN yum install -y \
+        cuda-samples-$CUDA_PKG_VERSION && \
+    rm -rf /var/cache/yum/*
+
+WORKDIR /usr/local/cuda/samples/0_Simple/vectorAdd
+RUN make
+
+CMD ./vectorAdd
diff --git a/samples/deviceQuery/Dockerfile b/samples/deviceQuery/Dockerfile
diff --git a/samples/matrixMulCUBLAS/Dockerfile b/samples/matrixMulCUBLAS/Dockerfile
diff --git a/samples/ubuntu/bandwidthTest/Dockerfile b/samples/ubuntu/bandwidthTest/Dockerfile
@@ -0,0 +1,10 @@
+FROM cuda
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+        cuda-samples-$CUDA_PKG_VERSION && \
+    rm -rf /var/lib/apt/lists/*
+
+WORKDIR /usr/local/cuda/samples/1_Utilities/bandwidthTest
+RUN make
+
+CMD ./bandwidthTest --mode=shmoo
diff --git a/samples/ubuntu/deviceQuery/Dockerfile b/samples/ubuntu/deviceQuery/Dockerfile
@@ -0,0 +1,10 @@
+FROM cuda
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+        cuda-samples-$CUDA_PKG_VERSION && \
+    rm -rf /var/lib/apt/lists/*
+
+WORKDIR /usr/local/cuda/samples/1_Utilities/deviceQuery
+RUN make
+
+CMD ./deviceQuery
diff --git a/samples/ubuntu/matrixMulCUBLAS/Dockerfile b/samples/ubuntu/matrixMulCUBLAS/Dockerfile
@@ -0,0 +1,10 @@
+FROM cuda
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+        cuda-samples-$CUDA_PKG_VERSION && \
+    rm -rf /var/lib/apt/lists/*
+
+WORKDIR /usr/local/cuda/samples/0_Simple/matrixMulCUBLAS
+RUN make
+
+CMD ./matrixMulCUBLAS -sizemult=10
diff --git a/samples/ubuntu/nvidia-smi/Dockerfile b/samples/ubuntu/nvidia-smi/Dockerfile
@@ -0,0 +1,3 @@
+FROM cuda
+
+CMD nvidia-smi -q
diff --git a/samples/ubuntu/vectorAdd/Dockerfile b/samples/ubuntu/vectorAdd/Dockerfile
@@ -0,0 +1,10 @@
+FROM cuda
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+        cuda-samples-$CUDA_PKG_VERSION && \
+    rm -rf /var/lib/apt/lists/*
+
+WORKDIR /usr/local/cuda/samples/0_Simple/vectorAdd
+RUN make
+
+CMD ./vectorAdd
diff --git a/samples/vectorAdd/Dockerfile b/samples/vectorAdd/Dockerfile
diff --git a/ubuntu-14.04/cuda/6.5/devel/Dockerfile b/ubuntu-14.04/cuda/6.5/devel/Dockerfile
@@ -0,0 +1,16 @@
+FROM cuda:6.5-runtime
+MAINTAINER NVIDIA CORPORATION <digits@nvidia.com>
+
+RUN apt-get update && apt-get install -y --no-install-recommends --force-yes \
+        cuda-core-$CUDA_PKG_VERSION \
+        cuda-misc-headers-$CUDA_PKG_VERSION \
+        cuda-command-line-tools-$CUDA_PKG_VERSION \
+        cuda-license-$CUDA_PKG_VERSION \
+        cuda-cublas-dev-$CUDA_PKG_VERSION \
+        cuda-cufft-dev-$CUDA_PKG_VERSION \
+        cuda-curand-dev-$CUDA_PKG_VERSION \
+        cuda-cusparse-dev-$CUDA_PKG_VERSION \
+        cuda-npp-dev-$CUDA_PKG_VERSION \
+        cuda-cudart-dev-$CUDA_PKG_VERSION \
+        cuda-driver-dev-$CUDA_PKG_VERSION && \
+    rm -rf /var/lib/apt/lists/*
diff --git a/ubuntu-14.04/cuda/6.5/Dockerfile → ubuntu-14.04/cuda/6.5/runtime/Dockerfile b/ubuntu-14.04/cuda/6.5/Dockerfile → ubuntu-14.04/cuda/6.5/runtime/Dockerfile
@@ -12,8 +12,15 @@ RUN apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/r
 ENV CUDA_VERSION 6.5
 LABEL com.nvidia.cuda.version="6.5"
 
+ENV CUDA_PKG_VERSION 6-5=6.5-19
 RUN apt-get update && apt-get install -y --no-install-recommends --force-yes \
-        cuda-toolkit-6-5=6.5-19 && \
+        cuda-cublas-$CUDA_PKG_VERSION \
+        cuda-cufft-$CUDA_PKG_VERSION \
+        cuda-curand-$CUDA_PKG_VERSION \
+        cuda-cusparse-$CUDA_PKG_VERSION \
+        cuda-npp-$CUDA_PKG_VERSION \
+        cuda-cudart-$CUDA_PKG_VERSION && \
+    ln -s cuda-$CUDA_VERSION /usr/local/cuda && \
     rm -rf /var/lib/apt/lists/*
 
 RUN echo "/usr/local/cuda/lib" >> /etc/ld.so.conf.d/cuda.conf && \

diff --git a/ubuntu-14.04/cuda/7.0/cudnn b/ubuntu-14.04/cuda/7.0/cudnn
diff --git a/ubuntu-14.04/cuda/7.0/devel/Dockerfile b/ubuntu-14.04/cuda/7.0/devel/Dockerfile
@@ -0,0 +1,18 @@
+FROM cuda:7.0-runtime
+MAINTAINER NVIDIA CORPORATION <digits@nvidia.com>
+
+RUN apt-get update && apt-get install -y --no-install-recommends --force-yes \
+        cuda-core-$CUDA_PKG_VERSION \
+        cuda-misc-headers-$CUDA_PKG_VERSION \
+        cuda-command-line-tools-$CUDA_PKG_VERSION \
+        cuda-license-$CUDA_PKG_VERSION \
+        cuda-nvrtc-dev-$CUDA_PKG_VERSION \
+        cuda-cusolver-dev-$CUDA_PKG_VERSION \
+        cuda-cublas-dev-$CUDA_PKG_VERSION \
+        cuda-cufft-dev-$CUDA_PKG_VERSION \
+        cuda-curand-dev-$CUDA_PKG_VERSION \
+        cuda-cusparse-dev-$CUDA_PKG_VERSION \
+        cuda-npp-dev-$CUDA_PKG_VERSION \
+        cuda-cudart-dev-$CUDA_PKG_VERSION \
+        cuda-driver-dev-$CUDA_PKG_VERSION && \
+    rm -rf /var/lib/apt/lists/*
diff --git a/ubuntu-14.04/cuda/7.0/devel/cudnn b/ubuntu-14.04/cuda/7.0/devel/cudnn
@@ -0,0 +1 @@
+cudnn2
diff --git a/ubuntu-14.04/cuda/7.0/cudnn-v2/Dockerfile → ...tu-14.04/cuda/7.0/devel/cudnn2/Dockerfile b/ubuntu-14.04/cuda/7.0/cudnn-v2/Dockerfile → ...tu-14.04/cuda/7.0/devel/cudnn2/Dockerfile
@@ -1,10 +1,11 @@
-FROM cuda:7.0
+FROM cuda:7.0-devel
 MAINTAINER NVIDIA CORPORATION <digits@nvidia.com>
 
 RUN apt-get update && apt-get install -y \
         curl && \
     rm -rf /var/lib/apt/lists/*
 
+ENV CUDNN_VERSION 2
 ENV CUDNN_DOWNLOAD_SUM 4b02cb6bf9dfa57f63bfff33e532f53e2c5a12f9f1a1b46e980e626a55f380aa
 
 RUN curl -fsSL http://developer.download.nvidia.com/compute/redist/cudnn/v2/cudnn-6.5-linux-x64-v2.tgz -O && \

diff --git a/ubuntu-14.04/cuda/7.0/Dockerfile → ubuntu-14.04/cuda/7.0/runtime/Dockerfile b/ubuntu-14.04/cuda/7.0/Dockerfile → ubuntu-14.04/cuda/7.0/runtime/Dockerfile
@@ -12,8 +12,17 @@ RUN apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/r
 ENV CUDA_VERSION 7.0
 LABEL com.nvidia.cuda.version="7.0"
 
+ENV CUDA_PKG_VERSION 7-0=7.0-28
 RUN apt-get update && apt-get install -y --no-install-recommends --force-yes \
-        cuda-toolkit-7-0=7.0-28 && \
+        cuda-nvrtc-$CUDA_PKG_VERSION \
+        cuda-cusolver-$CUDA_PKG_VERSION \
+        cuda-cublas-$CUDA_PKG_VERSION \
+        cuda-cufft-$CUDA_PKG_VERSION \
+        cuda-curand-$CUDA_PKG_VERSION \
+        cuda-cusparse-$CUDA_PKG_VERSION \
+        cuda-npp-$CUDA_PKG_VERSION \
+        cuda-cudart-$CUDA_PKG_VERSION && \
+    ln -s cuda-$CUDA_VERSION /usr/local/cuda && \
     rm -rf /var/lib/apt/lists/*
 
 RUN echo "/usr/local/cuda/lib" >> /etc/ld.so.conf.d/cuda.conf && \

diff --git a/ubuntu-14.04/cuda/7.0/runtime/cudnn b/ubuntu-14.04/cuda/7.0/runtime/cudnn
@@ -0,0 +1 @@
+cudnn2