-
-
Notifications
You must be signed in to change notification settings - Fork 11.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Audio cracking, robotic sound on MacOS with Meta Quest Pro (Related to Unity Best Latency setting) #4141
Comments
It is not correct to set them to the same value, it means that the audio output requests samples every second by block of 1 second, and it reads them in the audio buffer which attempts to keep on average 1 second of samples, it will often underrun. Firstly, if you record, is the recorded file (played with VLC for example) correct, with correct audio?
Then, does it work with only more audio buffering:
? If not, does it work better with a greater output buffer:
? |
Thanks for the swift reply. I ran the commands you suggested
But the problem is in all cases the same – at least I can't hear any significant improvements or differences. By the way the audio playback seems super slow, like a time streatch at 50% speed or so. BUT I just opened up other Quest apps, they work fine with all configurations of scrpy! So, unfortunately, it seems that the combination of my app (OpenSoundLab) and scrcpy does not work well together. OpenSoundLab is based on Unity and I guess I set the buffer size to the minimum setting there... will try to set it to normal and test that. |
Oh, so it might be a capture problem (stereo vs mono, or wrong bit-depth somewhere). Could you post the video file you recorded with |
Two results of https://drive.switch.ch/index.php/s/QiidYCHDf5wSmGs Haven't changed the app's audio buffer yet. |
OK, so ignore audio buffers, the problem is on the capture side (since it is recorded). Are you comfortable to build the project from sources? In particular, it would be interesting to get the results with this additional log: #4055 (comment) (but the provided binary was for client 2.0). |
Yes, I can try that. Do you have a theory? I find it weird that apparently all other Quest apps seem to work... |
I don't know, but it looks like this post (see the whole discussion).
Also try to increase |
Building went smooth, but you could add to the build instructions that Android Studio must be run once in order for it to write the sdk to ~/Library/Android/sdk. Here is a part of the log output:
I increased READ_MS to 20 but it did not help. Meta Quest home environment played without glitches, but my app had them again. I tried the suggested parameters again:
|
Just found out that at least one other Meta Quest app (Gorilla Tag) is also suffering from the glitches... |
There's a problem here: it gives 1920 samples every 20ms of pts, while at 48KHz it should give 960 samples every 20ms. Is it the same with the apps which work? |
Ah, I see. So here are some longer excerpts: Glitchy, within my app:
Glitchy, within Gorilla Tag:
Not glitchy, within home environment:
So every now and then the AudioRecorder is repeating a timestamp. I will try to increase the READ_MS even more. As a hotfix, I will try to catch these cases and skip the read when the pointer hasn't moved. What else could be tried? |
OK, so with longer excerpts, it looks ok (on average you get 20ms of samples every 20ms). If you record with some recorder app directly on the device, does it work better? |
It looked OK, but it did not sound OK... Increasing the READ_MS more did not help... The hotfix I was not able to get working correctly due to the complexity of the code... BUT: I just rebuilt my app with the "Good Latency" setting in Unity instead of "Best Latency" and the glitches are gone without any patches and with default parameters! Should I change the title of this ticket to indicate the source of the trouble? I assume all Unity apps can be affected. So, now some new questions arise, since I want to use scrcpy for capturing or streaming OpenSoundLab performances wirelessly: Is there a way to play the sound both on the headset and on the receiving computer simulataneously? Apparently not yet, but that would be great, see #3891 and #3875. Is there any chance to get to super low latencies? At the moment I really have to keep --audio-buffer at the default of 50 while on USB, lower values give dropouts. And even if I lower the settings, the audio always seems to have some minimum latency of at least 100ms, while the video can be very low latency. I am on wired speakers. For performance settings that is too high. Couldn't find a solution in the existing issues. |
Oh, interesting!
One problem is that With lower capture/encoding values, we could theoretically use a smaller audio buffer;
Paradoxically, if you lower the audio buffer and get buffer underruns, it will probably temporarily increase the latency (because you will "play" silence during the underrun + the real audio samples when they arrive, with compensation to speed them up).
Indeed, currently not possible 😞 |
Could you post the logs when READ_MS is 5ms (or even lower)? |
Yes, but do you mean with low or normal audio buffers in Unity? |
With "Best Latency" (when there are glitches). |
Output with 5ms:
Output with 2ms:
Output with 1ms:
|
Of course it would be great if the Best Latency setting could be glitch free as well... |
OK, so like all Android devices, even when we read blocks of 5ms, it gives 4 blocks of 5ms at a time every 20ms.
Did you test with some Android recording app? (there is probably a native recording feature on your system) Is the sound correctly recorded? And with sndcpy? |
The native recorder seems to record without glitches in the low latency setting (but is not really suitable for me since it is mono only). sndcpy I just tried for the first time, but doesn't play sound:
|
Is it a firewall thing? I am on a corporate laptop where I cannot edit the ports. |
There have been reports on cracking audio, some of them have been (partly) resolved:
#3793
#4055
This issue ticket here is also for audio crackling but for this specific configuration:
Environment
Describe the bug
I am mirroring my audio app OpenSoundLab. The image seems fine as usual, but audio is never clean and glitch free. Sounds like plenty of buffer underruns. I have increased both audio buffers up to seconds, but it doesnt change the glitches. Very low settings make it worse though. If I record the sound has the same glitch to it. adb via TCP/IP is a bit worse than USB.
I am speculating it might be something on the sender side?
The text was updated successfully, but these errors were encountered: