[BUG] Installing through pip causes cuda.ccudart' has no attribute '__pyx_capi__'

[zerothi]

Describe the bug
When importing rmm I get an error:

Python 3.11.7 (main, Feb 13 2024, 19:40:15) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
default imports: m = math, nb = numbers, np = numpy, pl = pathlib
>>> import rmm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/nicpa/test-rmm/lib/python3.11/site-packages/rmm/__init__.py", line 15, in <module>
    from rmm import mr
  File "/home/nicpa/test-rmm/lib/python3.11/site-packages/rmm/mr.py", line 14, in <module>
    from rmm._lib.memory_resource import (
  File "/home/nicpa/test-rmm/lib/python3.11/site-packages/rmm/_lib/__init__.py", line 15, in <module>
    from .device_buffer import DeviceBuffer
  File "device_buffer.pyx", line 1, in init rmm._lib.device_buffer
AttributeError: module 'cuda.ccudart' has no attribute '__pyx_capi__'
>>> 

Steps/Code to reproduce bug

I did this:

$> python3 -m venv test-rmm
$> source test-rmm/bin/activatte
$> pip install --extra-index-url=https://pypi.nvidia.com rmm-cu11
$> python
Python 3.11.7 (main, Feb 13 2024, 19:40:15) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rmm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/nicpa/test-rmm/lib/python3.11/site-packages/rmm/__init__.py", line 15, in <module>
    from rmm import mr
  File "/home/nicpa/test-rmm/lib/python3.11/site-packages/rmm/mr.py", line 14, in <module>
    from rmm._lib.memory_resource import (
  File "/home/nicpa/test-rmm/lib/python3.11/site-packages/rmm/_lib/__init__.py", line 15, in <module>
    from .device_buffer import DeviceBuffer
  File "device_buffer.pyx", line 1, in init rmm._lib.device_buffer
AttributeError: module 'cuda.ccudart' has no attribute '__pyx_capi__'
>>> 

Note, when following the conda installation guide, I got no problems.
Basically the conda installation was:

$> conda create -n test-rmm
$> conda activate test-rmm
$> conda install -c rapidsai -c conda-forge -c nvidia rmm cuda-version=12.0
$> python3
>>> import rmm
>>>

Hence, no problems there.

Expected behavior

It is unclear to me exactly why this happens, the best I can find is that the cython installed sources has a wrong path for the shipped
library, but I can't really see why it would be problematic since it works for conda (I suspect the installation scripts are roughly the same).
One thing that I found was this: cython/cython#3627

I believe the environment details are irrelevant here...
As this has to do with basic import statements, and nothing related to the actual environment.

I should note, we have tried this on two machines, one without GPU's, and one with a100 cards. Regardless the same error happens, so if it has to do with some env-vars not properly setup, please let us know (the print_env.sh didn't show any CUDA env vars...)

[leofang]

This is tracked in NVIDIA/cuda-python#215. We are working on it. For the time being please downgrade your cuda-python version as instructed there.

[zerothi]

Super, thanks! I didn't find this one! Feel free to close if you wish!

[leofang]

It is ok to keep it open until we resolve it.

[vyasr]

This was resolved by the last release of cuda-python.