SpectralCentroid transform

[turian]

🚀 Feature

torchaudio.transforms.SpectralCentroid

Motivation

This is a common audio transform, included in Librosa.

Pitch

A differentiable SpectralCentroid would be useful for audio neural networks.

Alternatives

Here is the librosa implementation.

Additional context

Here is a simplified implementation I have made, based upon the librosa implementation:

def spectral_centroid(y, sr, n_fft, hop_length=512):
    S = torchaudio.transforms.Spectrogram(n_fft=n_fft, hop_length=hop_length, power=1.0)(y)

#    freq = fft_frequencies(sr=sr, n_fft=n_fft)
    freq = torch.linspace(0,
                       float(sr) / 2,
                       int(1 + n_fft//2))
                       #endpoint=True)

    if freq.ndim == 1:
        freq = freq.reshape((-1, 1))

    def tl1norm(S):
        return S / torch.sum(torch.abs(S), axis=0)
        
    # Column-normalize S
    return torch.sum(freq * tl1norm(S),
                  axis=0, keepdims=True)

[jacobjwebber]

Hi,
I would use this if it existed! It would also be useful if it could be limited to a specific range of frequencies so that I could choose a specific (e.g. voiced) region of the spectrogram.
This could be done with f_min and f_max params which could default to 0 and sr/2 respectively. S could then be cropped to bins that correspond to frequencies within this range.

Jacob

[mthrok]

Hi @turian @jacobjwebber

Yes, looks like this is a good addition. Is either of you (or anyone reading this) interested in working this?

[NicolasHug]

I'll give this a shot!

[jacobjwebber]

Me too! happy to help, @NicolasHug

[mthrok]

Hi,
I would use this if it existed! It would also be useful if it could be limited to a specific range of frequencies so that I could choose a specific (e.g. voiced) region of the spectrogram.
This could be done with f_min and f_max params which could default to 0 and sr/2 respectively. S could then be cropped to bins that correspond to frequencies within this range.

Jacob

Hi @jacobjwebber

Please checkout #1167 for the SpectralCentroid implementation. For your suggestion on f_min and f_max, we decided to do it in separately so it will not be included in #1167. That way it's easier to see the changes from the base implementation.

[jacobjwebber]

Excellent, thanks @mthrok

I will make a new merge request with optional frequency limiting, probably this weekend.

[jacobjwebber]

Just had a look again --- Think it would be much easier to wait until #1167 is merged then do a completely separate one after with f_min / f_max cropping

[mthrok]

@jacobjwebber #1167 is merged. Feel free to open an issue or PR for the f_min / f_max cropping