Blank zeroed rows in Mel spectrograms

[duvtedudug]

🐛 Bug

Mel spectrogram have some random rows that are all zeros. Normal spectrograms are fine.

To Reproduce

Steps to reproduce the behavior:

1. mel = torchaudio.transforms.MelSpectrogram()(waveform)

or

1. spec = torchaudio.transforms.Spectrogram()(waveform)
2. mel = torchaudio.transforms.MelScale()(spec)

Both give something like this (white lines are zero valued)

Expected behavior

All rows to have data!

Environment

• Tried both pytorch 1.5 and 1.6

[mthrok]

Hi @duvtedudug

Thanks for reporting the issue. Can you provide us how you load audio file? Also, do you know a publicly accessible audio data that causes the same issue, or can you share your data?

[duvtedudug]

After more exploration the cause was having not enough n_fft bins in the spectrogram to get the default 128 Mel bins.

I was following the code in the tutorial

Here's my code...

filename = "my_audio.wav"
waveform, sample_rate = torchaudio.load(filename)

spec = torchaudio.transforms.Spectrogram()(waveform)

# But if I do this it works...
# spec = torchaudio.transforms.Spectrogram(n_fft=1024)(waveform)

plt.figure()
plt.imshow(torch.log2(spec[0,:,:]).numpy(), cmap='gray', origin='lower')

mel = torchaudio.transforms.MelScale()(spec)

plt.figure()
p = plt.imshow(torch.log2(mel[0,:,:]).numpy(), cmap='gray', origin='lower')

Just adding more resolution to the spectrogram seems to give the Mel conversion what it needs!

My only suggestion is to change the default n_fft values (or at least the values in the tutorial) so that it works out-of-the-box?

Thanks for your time!

[vincentqb]

After more exploration the cause was having not enough n_fft bins in the spectrogram to get the default 128 Mel bins.

I was following the code in the tutorial

Here's my code...

filename = "my_audio.wav"
waveform, sample_rate = torchaudio.load(filename)

spec = torchaudio.transforms.Spectrogram()(waveform)

# But if I do this it works...
# spec = torchaudio.transforms.Spectrogram(n_fft=1024)(waveform)

plt.figure()
plt.imshow(torch.log2(spec[0,:,:]).numpy(), cmap='gray', origin='lower')

mel = torchaudio.transforms.MelScale()(spec)

plt.figure()
p = plt.imshow(torch.log2(mel[0,:,:]).numpy(), cmap='gray', origin='lower')

Just adding more resolution to the spectrogram seems to give the Mel conversion what it needs!

My only suggestion is to change the default n_fft values (or at least the values in the tutorial) so that it works out-of-the-box?

Thanks for your time!

Thanks for the suggestion :) There was also a discussion in #384 about using powers of 2 instead. (I'm also noting that the default was changed in #83.)

If you use master, do you now get a warning about zero Mel filters if you don't specify n_fft? The warning was introduced in #914.