Different input frequencies are getting played as the same output frequency

[thehale]

In #2 I referenced a bug I was investigating. Here it is.

Background

I'm trying to use this library to prototype a personal data over sound project. I'm working on using the presence of a sound at any of 24 different frequencies to convey the data my application needs. Eventually I will be using several of these frequencies in parallel, which maps really well to this package's support of different tracks.

Issue

The issue I'm finding is that when I use this package to play a test sound at each of my chosen 24 frequencies I notice that some of the higher frequencies get played as the same tone.

Low Frequency Example

Here's the code:

from tones.mixer import Mixer
from tones import SINE_WAVE
from playsound import playsound

mixer = Mixer(44100, 0.5)

mixer.create_track(1, SINE_WAVE, attack=0.01, decay=0.1)

mixer.add_tone(1, 799.872020476724, .5)
mixer.add_tone(1, 899.604174163368, .5)
mixer.add_tone(1, 1000.40016006403, .5)
mixer.add_tone(1, 1100.5943209333, .5)
mixer.add_tone(1, 1301.06687483737, .5)
mixer.add_tone(1, 1400.56022408964, .5)
mixer.add_tone(1, 1500.60024009604, .5)
mixer.add_tone(1, 1601.53747597694, .5)
mixer.add_tone(1, 1799.20834832674, .5)
mixer.add_tone(1, 1899.69604863222, .5)
mixer.add_tone(1, 2000.80032012805, .5)
mixer.add_tone(1, 2100.84033613445, .5)
mixer.add_tone(1, 2296.73863114378, .5)
mixer.add_tone(1, 2396.93192713327, .5)
mixer.add_tone(1, 2497.5024975025, .5)
mixer.add_tone(1, 2597.4025974026, .5)
mixer.add_tone(1, 2801.12044817927, .5)
mixer.add_tone(1, 2903.60046457607, .5)
mixer.add_tone(1, 3001.20048019208, .5)
mixer.add_tone(1, 3105.5900621118, .5)
mixer.add_tone(1, 3306.87830687831, .5)
mixer.add_tone(1, 3401.36054421769, .5)
mixer.add_tone(1, 3501.40056022409, .5)
mixer.add_tone(1, 3607.50360750361, .5)

mixer.write_wav('tones.wav')
playsound('tones.wav')

And here's a spectrogram of the frequencies I'm getting as a result:

High Frequency Example

It appears as though playing higher frequencies exacerbates the problem. This code plays the same frequency intervals (not note intervals) but starting 3000 Hz higher than the previous example.

from tones.mixer import Mixer
from tones import SINE_WAVE
from playsound import playsound

mixer = Mixer(44100, 0.5)

mixer.create_track(1, SINE_WAVE, attack=0.01, decay=0.1)

mixer.add_tone(1, 3990.42298483639, .5)
mixer.add_tone(1, 4105.09031198686, .5)
mixer.add_tone(1, 4201.68067226891, .5)
mixer.add_tone(1, 4302.92598967298, .5)
mixer.add_tone(1, 4492.36298292902, .5)
mixer.add_tone(1, 4608.29493087558, .5)
mixer.add_tone(1, 4699.24812030075, .5)
mixer.add_tone(1, 4793.86385426654, .5)
mixer.add_tone(1, 4995.004995005, .5)
mixer.add_tone(1, 5102.04081632653, .5)
mixer.add_tone(1, 5213.76433785193, .5)
mixer.add_tone(1, 5291.00529100529, .5)
mixer.add_tone(1, 5494.50549450549, .5)
mixer.add_tone(1, 5580.35714285714, .5)
mixer.add_tone(1, 5714.28571428571, .5)
mixer.add_tone(1, 5807.20092915215, .5)
mixer.add_tone(1, 6002.40096038415, .5)
mixer.add_tone(1, 6105.0061050061, .5)
mixer.add_tone(1, 6211.1801242236, .5)
mixer.add_tone(1, 6321.11251580278, .5)
mixer.add_tone(1, 6493.50649350649, .5)
mixer.add_tone(1, 6613.75661375661, .5)
mixer.add_tone(1, 6675.56742323097, .5)
mixer.add_tone(1, 6802.72108843537, .5)

mixer.write_wav('tones.wav')
playsound('tones.wav')

Here's the resulting spectrogram.

Other Thoughts

I'm wondering if this has something to do with how this package focuses on playing specific notes (i.e. music composition). Perhaps my frequencies are simply getting rounded to the closest note? I didn't see anything that would seem to be doing that in the mixer.add_tone() function, but it's an idea that I've had nagging at me while I look through things.

I'm happy to help develop a solution to improve this great package, I'm just getting stuck when I tackle it on my own. Hoping @eriknyquist has some added insight into why this might be occurring.

[eriknyquist-avive]

@jhale1805 I think you hit the nail on the head when you said "I'm wondering if this has something to do with how this package focuses on playing specific notes (i.e. music composition)?"

I never did any sort of detailed testing of specific frequencies, like you are doing now. This module was very much intended for producing musical tones, and the extent of my testing was pretty much just using my ears to make sure things sound musically OK.

That being said, I will take a look at the code which handles specific frequency values, and see if there is an obvious problem that could cause such a loss of precision.

Thanks!

[eriknyquist-avive]

@jhale1805 I can possibly help speed up your investigation; the problem is most likely with the _sine_wave_table function here https://github.com/eriknyquist/tones/blob/master/tones/tone.py#L6

This function is called by the Tone.samples() function (right here https://github.com/eriknyquist/tones/blob/master/tones/tone.py#L197), to obtain a set of samples that make up a single 360 degree sine wave oscillation in the desired frequency/sample rate/amplitude (that is, a single period's worth of samples of the output sine wave waveform).

The Tone.samples() function then iterates over this table multiple times, as many times as is needed to create the number of samples we need for the requested note time.

My guess is that there is some significant loss of precision that occurs when I do period = int(rate / freq) in the _sine_wave_table function, and this is what's causing the anomaly you're seeing where the output seems to "snap" to certain frequencies.

I'm not sure exactly how I would resolve that, right now, but this is the area I'm drawn to right now based on your description.

[eriknyquist]

OK, so after I explained that to you I'm thinking that the problem is indeed _sine_wave_table, or more specifically, the approach of generating a single period's worth of sine wave samples and then duplicating it multiple times to get the desired note length.

This approach assumes that the full period of any sine wave at any frequency can be described by a discrete number of samples, when in reality, the full period of a sine wave is likely going to have some "fractional" sample at the end (e.g. a full period of 1555Hz, at 44100 sample rate, works out to 28.36 samples), unless the sine wave frequency happens to be an exact multiple of the sample rate.

This might also explain the weird harmonics reported in #1, since the issue I described above would result in sine waves that are not perfect-- there would be little "blips" in the waveform between every period, which I'm guessing would result in some odd harmonic content.

I think the correct way to do this would be to generate all samples for the full note length at once, instead of just doing a single period and then duplicating it, just like is being done in this stackoverflow answer;

https://stackoverflow.com/questions/8299303/generating-sine-wave-sound-in-python

I don't have time to work on it and test it right now (I can get to that next weekend), but I thought I would just dump that info here in case it helps you out.

[thehale]

I think you're on to something. I plotted the output waveform using this Stack Overflow post as a guide (https://stackoverflow.com/a/18625294) and got the following output (zoomed in a lot).

Just as you predicted, there is an odd change of slope at the end of each period that I don't think can be attributed to the minor imperfections introduced by using digital samples instead of an analog signal. In this example, it looks like the slope notably decreases at the end of each period, which would result in a slightly lower output frequency. This seems to illustrate how, as you eloquently said it, some input frequencies "snap" to the same output frequency.

I'll try out the solution you found and give another update in a bit.

Just for reference, the exact code that produced that image

from tones.mixer import Mixer
from tones import SINE_WAVE

mixer = Mixer(44100, 0.5)

mixer.create_track(1, SINE_WAVE, attack=0.01, decay=0.1)

mixer.add_tone(1, 2597.4025974026, .25)
mixer.add_tone(1, 2695.41778975741, .25)
mixer.add_tone(1, 2801.12044817927, .25)
mixer.add_tone(1, 3001.20048019208, .25)
mixer.add_tone(1, 3105.5900621118, .25)
mixer.add_tone(1, 3203.07495195388, .25)
mixer.add_tone(1, 3306.87830687831, .25)
mixer.add_tone(1, 3501.40056022409, .25)
mixer.add_tone(1, 3607.50360750361, .25)
mixer.add_tone(1, 3700.96225018505, .25)
mixer.add_tone(1, 3799.39209726444, .25)

mixer.write_wav('tones.wav')

#Addition
import wave
import numpy as np
import matplotlib.pyplot as plt

spf = wave.open('tones.wav')

signal = spf.readframes(-1)
signal = np.fromstring(signal, "Int16")

plt.plot(signal)
plt.show()
#/Addition

[thehale]

After some more tinkering it looks like your suggested solution works great!

The "Low Frequency" code from my original post now registers on the spectrogram as expected:

In addition to each distinct input frequency now getting its own distinct output, you'll notice that my original screenshot had a thin green line showing that the output frequency of the twelfth tone was 2203 Hz - a concerning 103 Hz off of the original ~2100 Hz. This new version now plays that same twelfth tone at 2109 Hz - only 9 Hz off of the original.
The harmonics are still present, but not as harshly as before.

The "High Frequency" example also works much better now:

And the output wave form also pretty much looks like a perfect sine wave.

You'll see that I submitted a merge request with my solution. As indicated there, I only tested for my specific use case, so you'll want to make appropriate updates to the other waveforms you support that I'm not as familiar with before re-publishing this package to pip.

Thanks again for this great package and your help with this issue!