Can this package be installed without a CUDA device?

[jorisvandenbossche]

I tried to install cudatoolkit-dev on a machine without GPU / CUDA enabled device (to test if I can compile something, without running), by creating a new clean env on an Ubuntu 18.04 laptop. The installation seems to work initially (no errors), but the toolkit actually doesn't seem to be installed.

Creation of the env:

``` (base) joris@joris-XPS-13-9380:~/scipy$ conda create -n test-cuda cudatoolkit-dev Collecting package metadata: done Solving environment: done

==> WARNING: A newer version of conda exists. <==
current version: 4.6.14
latest version: 4.7.5

Please update conda by running

$ conda update -n base conda

Package Plan

environment location: /home/joris/miniconda3/envs/test-cuda

added / updated specs:
- cudatoolkit-dev

The following packages will be downloaded:

package                    |            build
---------------------------|-----------------
sqlite-3.29.0              |       hcee41ef_0         2.0 MB  conda-forge
------------------------------------------------------------
                                       Total:         2.0 MB

The following NEW packages will be INSTALLED:

_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main
asn1crypto conda-forge/linux-64::asn1crypto-0.24.0-py37_1003
bzip2 conda-forge/linux-64::bzip2-1.0.6-h14c3975_1002
ca-certificates conda-forge/linux-64::ca-certificates-2019.6.16-hecc5488_0
certifi conda-forge/linux-64::certifi-2019.6.16-py37_0
cffi conda-forge/linux-64::cffi-1.12.3-py37h8022711_0
chardet conda-forge/linux-64::chardet-3.0.4-py37_1003
conda conda-forge/linux-64::conda-4.7.5-py37_0
conda-package-han~ conda-forge/linux-64::conda-package-handling-1.3.11-py37_0
cryptography conda-forge/linux-64::cryptography-2.7-py37h72c5cf5_0
cudatoolkit-dev conda-forge/linux-64::cudatoolkit-dev-10.0-1
icu conda-forge/linux-64::icu-58.2-hf484d3e_1000
idna conda-forge/linux-64::idna-2.8-py37_1000
libarchive conda-forge/linux-64::libarchive-3.3.3-hb44662c_1005
libffi conda-forge/linux-64::libffi-3.2.1-he1b5a44_1006
libgcc-ng pkgs/main/linux-64::libgcc-ng-9.1.0-hdf63c60_0
libiconv conda-forge/linux-64::libiconv-1.15-h516909a_1005
libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0
libxml2 conda-forge/linux-64::libxml2-2.9.9-h13577e0_1
lz4-c conda-forge/linux-64::lz4-c-1.8.3-he1b5a44_1001
lzo conda-forge/linux-64::lzo-2.10-h14c3975_1000
ncurses conda-forge/linux-64::ncurses-6.1-hf484d3e_1002
openssl conda-forge/linux-64::openssl-1.1.1c-h516909a_0
pip conda-forge/linux-64::pip-19.1.1-py37_0
pycosat conda-forge/linux-64::pycosat-0.6.3-py37h14c3975_1001
pycparser conda-forge/linux-64::pycparser-2.19-py37_1
pyopenssl conda-forge/linux-64::pyopenssl-19.0.0-py37_0
pysocks conda-forge/linux-64::pysocks-1.7.0-py37_0
python conda-forge/linux-64::python-3.7.3-h33d41f4_1
python-libarchive~ conda-forge/linux-64::python-libarchive-c-2.8-py37_1004
readline conda-forge/linux-64::readline-8.0-hf8c457e_0
requests conda-forge/linux-64::requests-2.22.0-py37_0
ruamel_yaml conda-forge/linux-64::ruamel_yaml-0.15.71-py37h14c3975_1000
setuptools conda-forge/linux-64::setuptools-41.0.1-py37_0
six conda-forge/linux-64::six-1.12.0-py37_1000
sqlite conda-forge/linux-64::sqlite-3.29.0-hcee41ef_0
tk conda-forge/linux-64::tk-8.6.9-hed695b0_1002
tqdm conda-forge/noarch::tqdm-4.32.2-py_0
urllib3 conda-forge/linux-64::urllib3-1.24.3-py37_0
wheel conda-forge/linux-64::wheel-0.33.4-py37_0
xz conda-forge/linux-64::xz-5.2.4-h14c3975_1001
yaml conda-forge/linux-64::yaml-0.1.7-h14c3975_1001
zlib conda-forge/linux-64::zlib-1.2.11-h14c3975_1004
zstd conda-forge/linux-64::zstd-1.4.0-h3b9ef0a_0

Proceed ([y]/n)?

Downloading and Extracting Packages
sqlite-3.29.0 | 2.0 MB | ######################################################################################################################################################################## | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

To activate this environment, use

$ conda activate test-cuda

To deactivate an active environment, use

$ conda deactivate

```

Activating the env already case some errors:

(base) joris@joris-XPS-13-9380:~/scipy$ act test-cuda
ln: target '/home/joris/miniconda3/envs/test-cuda/bin/zstdmt' is not a directory
ln: target '/home/joris/miniconda3/envs/test-cuda/lib/xml2Conf.sh' is not a directory
ln: target '/home/joris/miniconda3/envs/test-cuda/bin/zstdmt' is not a directory
ln: target '/home/joris/miniconda3/envs/test-cuda/lib/xml2Conf.sh' is not a directory
ln: target '/home/joris/miniconda3/envs/test-cuda/lib/xml2Conf.sh' is not a directory
ln: target '/home/joris/miniconda3/envs/test-cuda/include/zstd.h' is not a directory

After that, the nvcc compiler is not on the path, the pkgs/cudatoolkit-dev dir is empty except for a md5sum.txt file (although the cudatoolkit-dev-activate.sh script tries to link to bins located there).

So at least it seems the installation was not complete?

Environment (conda list):

$ conda list
# packages in environment at /home/joris/miniconda3/envs/test-cuda:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
asn1crypto                0.24.0                py37_1003    conda-forge
bzip2                     1.0.6             h14c3975_1002    conda-forge
ca-certificates           2019.6.16            hecc5488_0    conda-forge
certifi                   2019.6.16                py37_0    conda-forge
cffi                      1.12.3           py37h8022711_0    conda-forge
chardet                   3.0.4                 py37_1003    conda-forge
conda                     4.7.5                    py37_0    conda-forge
conda-package-handling    1.3.11                   py37_0    conda-forge
cryptography              2.7              py37h72c5cf5_0    conda-forge
cudatoolkit-dev           10.0                          1    conda-forge
icu                       58.2              hf484d3e_1000    conda-forge
idna                      2.8                   py37_1000    conda-forge
libarchive                3.3.3             hb44662c_1005    conda-forge
libffi                    3.2.1             he1b5a44_1006    conda-forge
libgcc-ng                 9.1.0                hdf63c60_0  
libiconv                  1.15              h516909a_1005    conda-forge
libstdcxx-ng              9.1.0                hdf63c60_0  
libxml2                   2.9.9                h13577e0_1    conda-forge
lz4-c                     1.8.3             he1b5a44_1001    conda-forge
lzo                       2.10              h14c3975_1000    conda-forge
ncurses                   6.1               hf484d3e_1002    conda-forge
openssl                   1.1.1c               h516909a_0    conda-forge
pip                       19.1.1                   py37_0    conda-forge
pycosat                   0.6.3           py37h14c3975_1001    conda-forge
pycparser                 2.19                     py37_1    conda-forge
pyopenssl                 19.0.0                   py37_0    conda-forge
pysocks                   1.7.0                    py37_0    conda-forge
python                    3.7.3                h33d41f4_1    conda-forge
python-libarchive-c       2.8                   py37_1004    conda-forge
readline                  8.0                  hf8c457e_0    conda-forge
requests                  2.22.0                   py37_0    conda-forge
ruamel_yaml               0.15.71         py37h14c3975_1000    conda-forge
setuptools                41.0.1                   py37_0    conda-forge
six                       1.12.0                py37_1000    conda-forge
sqlite                    3.29.0               hcee41ef_0    conda-forge
tk                        8.6.9             hed695b0_1002    conda-forge
tqdm                      4.32.2                     py_0    conda-forge
urllib3                   1.24.3                   py37_0    conda-forge
wheel                     0.33.4                   py37_0    conda-forge
xz                        5.2.4             h14c3975_1001    conda-forge
yaml                      0.1.7             h14c3975_1001    conda-forge
zlib                      1.2.11            h14c3975_1004    conda-forge
zstd                      1.4.0                h3b9ef0a_0    conda-forge

Details about conda and system ( conda info ):

$ conda info

     active environment : test-cuda
    active env location : /home/joris/miniconda3/envs/test-cuda
            shell level : 2
       user config file : /home/joris/.condarc
 populated config files : /home/joris/.condarc
          conda version : 4.6.14
    conda-build version : not installed
         python version : 3.7.3.final.0
       base environment : /home/joris/miniconda3  (writable)
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/joris/miniconda3/pkgs
                          /home/joris/.conda/pkgs
       envs directories : /home/joris/miniconda3/envs
                          /home/joris/.conda/envs
               platform : linux-64
             user-agent : conda/4.6.14 requests/2.22.0 CPython/3.7.3 Linux/4.15.0-1043-oem ubuntu/18.04.2 glibc/2.27
                UID:GID : 1001:1001
             netrc file : None
           offline mode : False

[andersy005]

As stated in the README

This package consists of a post-install script that downloads and installs the full cuda toolkit(compiler, libraries, with the exception of cuda drivers).

A cuda device with cuda drivers is needed for proper functionality.

[jorisvandenbossche]

Thanks for the answer!

Now, to me (but I am a CUDA / GPU noob) the README doesn't make that really clear. I know a cuda device is needed for all functionality, but in principle you can compile with nvcc without having a cuda device (AFAIK).
And I am also wondering, could the post install script raise an error if does not succeed? That might make it less confusing

[andersy005]

I concur. The post install script could use some improvements in error handling.

[andersy005]

I know a cuda device is needed for all functionality, but in principle you can compile with nvcc without having a cuda device (AFAIK).

@jorisvandenbossche, you are right. It's possible to use this package without a CUDA device, however, if you ever need functionality that requires access to cuda driver, it won't work.

[pearu]

A case, where installing cudatoolkit-dev to machine without a CUDA device would be needed, is building third-party libraries such as omniscidb-cuda (see conda-forge/staged-recipes#9225 ) that use nvcc, in conda-forge CI.

What would be needed to fix this issue?

[PythonNut]

Another use case would in a HPC environment where you install the package on a build node with no GPU, then run it on a compute node with no internet.