Do I have to build my images from the NVIDIA ones?

[tomjaguarpaw]

I have built an image based on an arbitrary non-NVIDIA image (it just so happens to be bamos/openface, if that's relevant). However, when I run it with

$ nvidia-docker run  -t -i ca6883e46289 /bin/bash

I still cannot see libcuda.so.1 in the image. In fact, find . -name libcuda.so.1 returns no results.

Do I have to build my image on one of the NVIDIA ones to make libcuda.so.1 accessible inside my container? If so, why? If not, what is going on?

[tomjaguarpaw]

This seems to be somewhat related to

• libcuda.so can not be found in /usr/local/cuda/lib64 when building mxnet in nvidia/cuda docker #37

• Couldn't open CUDA library libcuda.so.1 in when building using nvidia docker cuda8 image tensorflow/tensorflow#4931

[flx42]

Our official images have a special label that nvidia-docker will detect and then mount the driver files. This is explained on our wiki.
Try adding the following to your Dockerfile:

LABEL com.nvidia.volumes.needed="nvidia_driver"
ENV PATH /usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH}
ENV LD_LIBRARY_PATH /usr/local/nvidia/lib:/usr/local/nvidia/lib64

[tomjaguarpaw]

Interesting. Why do you require the presence of this flag? Isn't the fact that the user run nvidia-docker instead of docker evidence enough that you should link in the NVidia stuff?

[flx42]

Not really, to integrate into the ecosystem, you might need to swap the docker binary by nvidia-docker for all containers being ran. In this case we want to be a no-op if this is not a CUDA image.

[tomjaguarpaw]

How about a flag to force mounting the driver files even if it is not detected to be a CUDA image?

[flx42]

This will be possible with nvidia-docker 2.0.

[tomjaguarpaw]

OK thanks.

[gemfield]

Please be noted that these are not true any more with the latest docker and nvidia-docker.

[KyleRAnderson]

@gemfield do you happen to know what this would look like if we did want this working with the latest nvidia-docker2? I'm trying to get Pytorch working in a custom built image, seems like https://github.com/pytorch/pytorch/blob/master/Dockerfile uses the com.nvidia.volumes.needed label and I wanted to make sure it was the recommended way of handling this.