Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot initialize CUDA even if nvidia-smi detected #1813

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

fri-K
Copy link
Contributor

@fri-K fri-K commented Nov 19, 2024

CUDA can't be properly initialized if cuda-compat on host and docker mismatches. Removing cuda-compat-11-2 package fixes the issue

Issue discussion:
https://community.opendronemap.org/t/opendronemap-nodeodm-gpu-nvidia-smi-detected-cannot-initialize-cuda/21124
log:

[INFO] nvidia-smi detected
[INFO] Using CPU for feature extraction: Cannot initialize CUDA

CUDA can't be properly initialized if cuda-compat on host and docker mismatches. Removing cuda-compat-11-2 package fixes the issue

Issue discussion:
https://community.opendronemap.org/t/opendronemap-nodeodm-gpu-nvidia-smi-detected-cannot-initialize-cuda/21124
[INFO] nvidia-smi detected
[INFO] Using CPU for feature extraction: Cannot initialize CUDA
@pierotofy
Copy link
Member

pierotofy commented Nov 19, 2024

Thanks for the PR. Is this going to break for people using 11.2?

@fri-K
Copy link
Contributor Author

fri-K commented Nov 19, 2024

Thanks for the PR. Is this going to break for people using 11.2?

Hi, I hope not, but I tested it with 12.2, 12.4 and 12.6. I think I can test it with 11.2 in some days if you need me to?

@pierotofy
Copy link
Member

Pinging @ehallein to see what testing should be done here.

@ehallein
Copy link
Collaborator

Yep, will need to be tested on 11.2 if we want have that as the minimum requirement. What host operating system has this been tested on?

@fri-K
Copy link
Contributor Author

fri-K commented Nov 20, 2024

I tested on Ubuntu 22.04 and 24.04. I don't know what OS was cjgars using for his test

@ehallein
Copy link
Collaborator

I tested on Ubuntu 22.04 and 24.04. I don't know what OS was cjgars using for his test

Probably just needs to be tested on windows. If the GPU container is building I can test it, but won't be able to for a week as away.

@ehallein
Copy link
Collaborator

built docker image and tested Aukerman dataset on windows 11 with an RTX 3070, GPU was detected and everything worked

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants