Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It seems that the current version of bitsandbytes is not compatible with my CUDA 12.4 library. #1409

Open
Yukiiceeee opened this issue Nov 8, 2024 · 1 comment

Comments

@Yukiiceeee
Copy link

Yukiiceeee commented Nov 8, 2024

System Info

CUDA12.4

Reproduction

When using bitsandbytes, I noticed that it requires a libcusparse.so.11 file. However, my CUDA version is 12.4, which only includes libcusparse.so.12 but not libcusparse.so.11, resulting in an error. Isn’t bitsandbytes supposed to support CUDA 12.4? After searching for solutions, I received feedback suggesting that I should downgrade my CUDA version. Why is this issue happening?

The errors are as follows:

warnings.warn(
>> world size:1; torch rank:0; OS local rank:0; device:cuda:0;
Could not find the bitsandbytes CUDA binary at PosixPath('/d1/.conda/newcmoe/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda118.so')
Could not load bitsandbytes native library: /d1/.conda/newcmoe/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "/d1/.conda/newcmoe/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 104, in <module>
    lib = get_native_library()
  File "/d1/.conda/newcmoe/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 91, in get_native_library
    dll = ct.cdll.LoadLibrary(str(binary_path))
  File "/d1/.conda/newcmoe/lib/python3.10/ctypes/__init__.py", line 452, in LoadLibrary
    return self._dlltype(name)
  File "/d1/.conda/newcmoe/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /d1/.conda/newcmoe/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so: cannot open shared object file: No such file or directory

CUDA Setup failed despite CUDA being available. Please run the following command to get more information:

python -m bitsandbytes

Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

Expected behavior

It should be able for CUDA12.4.

@matthewdouglas
Copy link
Member

It would appear that you have a PyTorch installation built with CUDA 11.8. It should work if you upgrade to a CUDA 12.4 build of PyTorch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants