got soundfile.LibsndfileError: <unprintable LibsndfileError object> when start training

[Stillalive-myj]

❓ Questions

I start the command "dora run variant=myj" and got this
I don't have a NVIDIA GPU, so this is just a test
Error executing job with overrides: ['variant=myj']

All outputs here

C:\Users\Stillalive-myj\demucs-main\demucs\train.py:21: UserWarning: TorchAudio's global backend is now deprecated. Please enable distpatcher by setting TORCHAUDIO_USE_BACKEND_DISPATCHER=1, and specify backend when calling load/info/save function.
import torchaudio
C:\Users\Stillalive-myj.conda\envs\demucs\lib\site-packages\hydra_internal\hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.
ret = run_job(
[06-01 19:51:47][demucs.train][INFO] - For logs, checkpoints and samples check C:\Users\Stillalive-myj\demucs-main\outputs\xps\c1d4f44e
[06-01 19:51:47][dora.distrib][INFO] - world_size is 1, skipping init.
[06-01 19:51:48][demucs.train][INFO] - train/valid set size: 18368 14
[06-01 19:51:48][demucs.solver][INFO] - ----------------------------------------------------------------------
[06-01 19:51:48][demucs.solver][INFO] - Training...
C:\Users\Stillalive-myj\demucs-main\demucs\repitch.py:14: UserWarning: TorchAudio's global backend is now deprecated. Please enable distpatcher by setting TORCHAUDIO_USE_BACKEND_DISPATCHER=1, and specify backend when calling load/info/save function.
import torchaudio as ta
C:\Users\Stillalive-myj\demucs-main\demucs\repitch.py:14: UserWarning: TorchAudio's global backend is now deprecated. Please enable distpatcher by setting TORCHAUDIO_USE_BACKEND_DISPATCHER=1, and specify backend when calling load/info/save function.
import torchaudio as ta
C:\Users\Stillalive-myj\demucs-main\demucs\repitch.py:14: UserWarning: TorchAudio's global backend is now deprecated. Please enable distpatcher by setting TORCHAUDIO_USE_BACKEND_DISPATCHER=1, and specify backend when calling load/info/save function.
import torchaudio as ta
C:\Users\Stillalive-myj\demucs-main\demucs\repitch.py:14: UserWarning: TorchAudio's global backend is now deprecated. Please enable distpatcher by setting TORCHAUDIO_USE_BACKEND_DISPATCHER=1, and specify backend when calling load/info/save function.
import torchaudio as ta
C:\Users\Stillalive-myj\demucs-main\demucs\repitch.py:14: UserWarning: TorchAudio's global backend is now deprecated. Please enable distpatcher by setting TORCHAUDIO_USE_BACKEND_DISPATCHER=1, and specify backend when calling load/info/save function.
import torchaudio as ta
C:\Users\Stillalive-myj\demucs-main\demucs\repitch.py:14: UserWarning: TorchAudio's global backend is now deprecated. Please enable distpatcher by setting TORCHAUDIO_USE_BACKEND_DISPATCHER=1, and specify backend when calling load/info/save function.
import torchaudio as ta
C:\Users\Stillalive-myj\demucs-main\demucs\repitch.py:14: UserWarning: TorchAudio's global backend is now deprecated. Please enable distpatcher by setting TORCHAUDIO_USE_BACKEND_DISPATCHER=1, and specify backend when calling load/info/save function.
import torchaudio as ta
C:\Users\Stillalive-myj\demucs-main\demucs\repitch.py:14: UserWarning: TorchAudio's global backend is now deprecated. Please enable distpatcher by setting TORCHAUDIO_USE_BACKEND_DISPATCHER=1, and specify backend when calling load/info/save function.
import torchaudio as ta
C:\Users\Stillalive-myj\demucs-main\demucs\repitch.py:14: UserWarning: TorchAudio's global backend is now deprecated. Please enable distpatcher by setting TORCHAUDIO_USE_BACKEND_DISPATCHER=1, and specify backend when calling load/info/save function.
import torchaudio as ta
C:\Users\Stillalive-myj\demucs-main\demucs\repitch.py:14: UserWarning: TorchAudio's global backend is now deprecated. Please enable distpatcher by setting TORCHAUDIO_USE_BACKEND_DISPATCHER=1, and specify backend when calling load/info/save function.
import torchaudio as ta
Error executing job with overrides: ['variant=myj']
Traceback (most recent call last):
File "C:\Users\Stillalive-myj\demucs-main\demucs\train.py", line 244, in main
solver.train()
File "C:\Users\Stillalive-myj\demucs-main\demucs\solver.py", line 195, in train
metrics['train'] = self._run_one_epoch(epoch)
File "C:\Users\Stillalive-myj\demucs-main\demucs\solver.py", line 306, in _run_one_epoch
for idx, sources in enumerate(logprog):
File "C:\Users\Stillalive-myj.conda\envs\demucs\lib\site-packages\dora\log.py", line 70, in next
value = next(self._iterator)
File "C:\Users\Stillalive-myj.conda\envs\demucs\lib\site-packages\torch\utils\data\dataloader.py", line 630, in next
data = self._next_data()
File "C:\Users\Stillalive-myj.conda\envs\demucs\lib\site-packages\torch\utils\data\dataloader.py", line 1345, in _next_data
return self._process_data(data)
File "C:\Users\Stillalive-myj.conda\envs\demucs\lib\site-packages\torch\utils\data\dataloader.py", line 1371, in _process_data
data.reraise()
File "C:\Users\Stillalive-myj.conda\envs\demucs\lib\site-packages\torch_utils.py", line 694, in reraise
raise exception
soundfile.LibsndfileError:

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

[Stillalive-myj]

may be a bug?

[CarlGao4]

Please provide more information by running:
python -c "import torch,torchaudio,soundfile,sys;print(sys.version,torch.__version__,torchaudio.__version__,soundfile.__version__,soundfile.__libsndfile_version__,sep='\n')"

[Stillalive-myj]

Please provide more information by running: python -c "import torch,torchaudio,soundfile,sys;print(sys.version,torch.__version__,torchaudio.__version__,soundfile.__version__,soundfile.__libsndfile_version__,sep='\n')"

3.9.19 | packaged by conda-forge | (main, Mar 20 2024, 12:38:46) [MSC v.1929 64 bit (AMD64)]
2.1.2+cpu
2.1.2+cpu
0.12.1
1.2.0

[zingoBexley]

[08-01 11:40:12][demucs.solver][INFO] - ----------------------------------------------------------------------
[08-01 11:40:12][demucs.solver][INFO] - Cross validation...
[08-01 11:40:13][demucs.solver][INFO] - Valid | Epoch 20 | 2/10 | 6.16 it/sec | Loss 0.3418 | Reco 0.3418 | Nsdr -2.129
[08-01 11:40:13][demucs.solver][INFO] - Valid | Epoch 20 | 4/10 | 6.72 it/sec | Loss 0.2954 | Reco 0.2954 | Nsdr -0.747
[08-01 11:40:13][demucs.solver][INFO] - Valid | Epoch 20 | 6/10 | 8.25 it/sec | Loss 0.3085 | Reco 0.3085 | Nsdr -0.068
[08-01 11:40:13][demucs.solver][INFO] - Valid | Epoch 20 | 8/10 | 8.55 it/sec | Loss 0.3263 | Reco 0.3263 | Nsdr -0.058
[08-01 11:40:13][demucs.solver][INFO] - Valid Summary | Epoch 20 | Loss=0.3411 | Reco=0.3411 | Nsdr=-0.564 | Best=0.3363 | Bname=main
[08-01 11:40:13][demucs.solver][INFO] - ----------------------------------------------------------------------
[08-01 11:40:13][demucs.solver][INFO] - Evaluating on the test set...
Error executing job with overrides: ['variant=upmix']
Traceback (most recent call last):
  File "/user/b/bsteers/ccrma_upmix_fzs/ccrma_upmix/demucs/train.py", line 243, in main
    solver.train()
  File "/user/b/bsteers/ccrma_upmix_fzs/ccrma_upmix/demucs/solver.py", line 279, in train
    metrics['test'] = evaluate(self, compute_sdr=compute_sdr)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/user/b/bsteers/ccrma_upmix_fzs/ccrma_upmix/demucs/evaluate.py", line 158, in evaluate
    metric_names = next(iter(all_tracks.values()))[model.sources[0]]
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
Executor: Worker 0 died, killing all workers

This issue I am running into seems very related. Anyone have an idea how to get through this?

[ShreyasxAnand]

I have the same issue with the Libsnfile error with an Nvidia GPU and CUDA 11.7

These are my results from the python version command

3.8.19 | packaged by conda-forge | (default, Mar 20 2024, 12:38:07) [MSC v.1929 64 bit (AMD64)]
1.13.1+cu117
0.13.1+cu117
0.11.0
1.2.2

[CarlGao4]

@zingoBexley It seems that you are encountering the evaluating process but has no data to pass in. You can just comment out the codes relating to evaluation.