Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RTX 3080 compatibility OSError: libcudart.so.10.2: cannot open shared object file: No such file or directory #2645

Closed
3 of 4 tasks
nairouzshehata opened this issue May 27, 2021 · 5 comments

Comments

@nairouzshehata
Copy link

📚 Installation

Environment

Checklist

  • I followed the installation guide.
  • I cannot find my error message in the FAQ.
  • I set up CUDA correctly and can compile CUDA code via nvcc.
  • I do have multiple CUDA versions on my machine.

Additional context

nvidia-smi shows cuda version 11.2 and I tried 10.2 here and got this message when trying to use GPU.

torch/cuda/__init__.py:104: UserWarning: 
GeForce RTX 3080 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_61 sm_70 sm_75 compute_37.
If you want to use the GeForce RTX 3080 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/
  warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))

I then tried 11.1 but now on import I'm getting the below

Torch version: 1.8.1
Cuda available: True
Cuda version: 11.1
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-9-f74940949094> in <module>
      3 print(f"Cuda version: {torch.version.cuda}")
      4 
----> 5 import torch_geometric
      6 print(f"Torch geometric version: {torch_geometric.__version__}")
/vol/biomedic3/nsm116/pygeo3/lib/python3.8/site-packages/torch_geometric/__init__.py in <module>
      3 
      4 from .debug import is_debug_enabled, debug, set_debug
----> 5 import torch_geometric.data
      6 import torch_geometric.transforms
      7 import torch_geometric.utils
/vol/biomedic3/nsm116/pygeo3/lib/python3.8/site-packages/torch_geometric/data/__init__.py in <module>
----> 1 from .data import Data
      2 from .temporal import TemporalData
      3 from .batch import Batch
      4 from .dataset import Dataset
      5 from .in_memory_dataset import InMemoryDataset
/vol/biomedic3/nsm116/pygeo3/lib/python3.8/site-packages/torch_geometric/data/data.py in <module>
      6 import torch
      7 import torch_geometric
----> 8 from torch_sparse import coalesce, SparseTensor
      9 from torch_geometric.utils import (contains_isolated_nodes,
     10                                    contains_self_loops, is_undirected)
/vol/biomedic3/nsm116/pygeo3/lib/python3.8/site-packages/torch_sparse/__init__.py in <module>
     12         '_saint', '_sample', '_relabel'
     13 ]:
---> 14     torch.ops.load_library(importlib.machinery.PathFinder().find_spec(
     15         f'{library}_{suffix}', [osp.dirname(__file__)]).origin)
     16 
/vol/biomedic3/nsm116/pygeo3/lib/python3.8/site-packages/torch/_ops.py in load_library(self, path)
    102             # static (global) initialization code in order to register custom
    103             # operators with the JIT.
--> 104             ctypes.CDLL(path)
    105         self.loaded_libraries.add(path)
    106 
/vol/biomedic3/nsm116/pygeo3/lib/python3.8/ctypes/__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error, winmode)
    379 
    380         if handle is None:
--> 381             self._handle = _dlopen(self._name, mode)
    382         else:
    383             self._handle = handle
OSError: libcudart.so.10.2: cannot open shared object file: No such file or directory

even after

export PATH=/vol/cuda/11.1.0-cudnn8.0.4.30/bin:$PATH
export CPATH=/vol/cuda/11.1.0-cudnn8.0.4.30/include:$CPATH
export LD_LIBRARY_PATH=/vol/cuda/11.1.0-cudnn8.0.4.30/lib64:$LD_LIBRARY_PATH

No wheels for nightly PyTorch

Any suggestions please?

@scarpma
Copy link
Contributor

scarpma commented May 28, 2021

Even if this is not a solution to your problem, I think that installing through anaconda is easier because it downloads everything that is needed. I have never found a problem in various ubuntu versions.

@rusty1s
Copy link
Member

rusty1s commented May 28, 2021

Somehow, PyTorch is still looking for CUDA 10.2 when loading the extension package. This might be fixable by also setting $CUDA_HOME to your CUDA 11.1 path.

@nairouzshehata
Copy link
Author

Even if this is not a solution to your problem, I think that installing through anaconda is easier because it downloads everything that is needed. I have never found a problem in various ubuntu versions.

thanks @scarpma yeah so I'm using conda and -c anaconda if a package is not available, I try conda-forge or other and my last resort is pip. However for pytorch-geometric I installed using the wheels as indicated on the docs.

@scarpma
Copy link
Contributor

scarpma commented May 31, 2021

Oh, sorry, I didn't know what a wheel is. By the way I use rtx 3090 and it works fine. Sorry I can't help...

@nairouzshehata
Copy link
Author

nairouzshehata commented May 31, 2021

Somehow, PyTorch is still looking for CUDA 10.2 when loading the extension package. This might be fixable by also setting $CUDA_HOME to your CUDA 11.1 path.

I created a fresh environment (installed packages as described to @scarpma) and instead of setting the paths I run the cuda setup.sh and all is good now

Torch version: 1.8.0
Cuda version: 11.1
Torch geometric version: 1.7.0
Python: 3.8.10

Thanks @rusty1s and @scarpma !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants