Need help on NEMO ASR ClusteringDiarizer!

[Suma-Rajashankar]

When I run the following code:

from nemo.collections.asr.models import ClusteringDiarizer
oracle_vad_clusdiar_model = ClusteringDiarizer(cfg=config)

I get the following error:

File ~/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:298, in _lazy_init()
    296 if "CUDA_MODULE_LOADING" not in os.environ:
    297     os.environ["CUDA_MODULE_LOADING"] = "LAZY"
--> 298 torch._C._cuda_init()
    299 # Some of the queued calls may reentrantly call _lazy_init();
    300 # we need to just return without initializing in that case.
    301 # However, we must not let any *other* threads in!
    302 _tls.is_initializing = True

RuntimeError: No HIP GPUs are available

How to fix this? I have installed pytorch (right version) and working in the GPU environment. Any help would be appreciated.

[tango4j]

Can you first check if you setup your GPU environment correctly ?
in python
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
or in bash
export CUDA_VISIBLE_DEVICES="0"

In addition,
Can you also specify:
(1) Model of NVIDIA GPU (e.g., RTX A6000)
(2) CUDA version ( type nvcc -V in CLI)
(3) NeMo toolkit version

to figure out the problem.

[eduardo-onate]

Hello,

I'm having a similar issue. I'm following the NeMo tutorial of Speaker Recognition Verification in my machine.

I'd assume my GPU environment is working correctly: torch.cuda.is_available() outputs True and I've set the trainer's accelerator as:

accelerator = 'gpu' if torch.cuda.is_available() else 'cpu'
config.trainer.accelerator = accelerator

Moreover, when I set my PyTorch Lightning trainer with trainer = pl.Trainer(**config.trainer) I get:

GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
`Trainer(val_check_interval=1.0)` was configured so validation will run at the end of the training epoch..

But then, while defining my speaker model speaker_model = nemo_asr.models.EncDecSpeakerLabelModel(cfg=config.model, trainer=trainer) I get the same error as Ms. Rajashankar:

File [/opt/conda/lib/python3.10/site-packages/torch/cuda/__init__.py:769](https://vscode-remote+ssh-002dremote-002bspeaker-002drecognition.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/torch/cuda/__init__.py:769), in current_device()
    767 def current_device() -> int:
    768     r"""Returns the index of a currently selected device."""
...
    300 # we need to just return without initializing in that case.
    301 # However, we must not let any *other* threads in!
    302 _tls.is_initializing = True

RuntimeError: No HIP GPUs are available

In my case, the previous keeps happening when I set os.environ["CUDA_VISIBLE_DEVICES"] = "0"

---

I'm leaving here the same information that was asked to Ms. Rajashankar:

(1) NVIDIA A10G
(2)

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Jun__8_16:49:14_PDT_2022
Cuda compilation tools, release 11.7, V11.7.99
Build cuda_11.7.r11.7/compiler.31442593_0

(3) Following the mentioned tutorial, I'm installing:

## Install NeMo
BRANCH = 'r1.21.0'
!python -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[asr]

Thanks in advance.