Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

high frequency artifacts in spectrograms with reduced max_freq #132

Closed
egitto opened this issue Jan 22, 2020 · 2 comments
Closed

high frequency artifacts in spectrograms with reduced max_freq #132

egitto opened this issue Jan 22, 2020 · 2 comments

Comments

@egitto
Copy link

egitto commented Jan 22, 2020

image

Steps to reproduce:

  1. Open Friture
  2. Reduce the 2D spectrogram's max frequency to 1000 Hz
  3. With a descending tone, speak a vowel with a lot of brightness, like an æ ("aaaa") sound
  4. Observe that (some of?) your harmonics mirror, and go up and down all over the place, instead of only floating above and parallel to your f0 like harmonics should

I understand that this is due to the decimator using the "simplest way"

    def decimate(self, samples):
        # first we remove as much points as possible
        if self.decimation > 1:
            samples.shape = len(samples) // self.decimation, self.decimation
            # the full way
            # samples = samples.mean(axis=1)
            # the simplest way
            samples = samples[:, 0]
        return samples

I propose using the full way rather than the simplest way, as that should mostly eliminate this problem (as the higher-than-threshold frequencies should approximately cancel themselves out), while maintaining the presumable performance benefit of giving a smaller set of samples to the FFT.

At the moment, the decimator seems to be optimizing performance when the window is cropped, but at the cost that the spectrogram has ghosts if there are significant harmonics above some threshold frequency. I expect that in most use cases of the app, there will be significant harmonics; human vocalizations and musical instruments are full of them; so probably this is also the case for times when one would want to crop the window?

I have created a PR for this. Some additional details including costs and benefits are there. #133

@egitto egitto changed the title when cropped, spectrogram does weird things to harmonics high frequency artifacts in spectrograms with reduced max_freq Jan 23, 2020
@tlecomte
Copy link
Owner

Thanks @egitto for this very clear report ! Obviously we have a sampling problem... As you found, decimating without pre-filtering is a perfect recipe for artifacts... I guess I wrote that part when I was not clear enough on sampling theory!

I'll try to look at whether there is a fast-enough and correct-enough way to do that. Using an average as you did in #133 is a step in the proper direction, but we might be able to do better (by using a filter to do the downsampling instead of an average). I don't have a lot of spare time these days, so I can't promise any ETA unfortunately.

tlecomte added a commit that referenced this issue Feb 16, 2020
Addresse #132. When the FFT spectrum is cropped by lowering the max frequency, the signal was decimated N times by taking one sample every N samples. This is not correct, produces many artifacts. To keep decimation, we would need to use a proper low-pass filter (as implemented for the octave spectum for example), but it does not really seem worth it. We will just apply the FFT on top of the undecimated signal.
@tlecomte
Copy link
Owner

I've removed the decimation completely in #137
Thanks again for the report.
Please reopen the issue if it does not work ! Thanks !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants