Audio crackling really bad #4055
Comments
|
Is the sound ok in the recorded file when you record with |
I tried scrcpy --record=file.mp4 , and it reported an error, like this D:\Program Files\scrcpy-win64>scrcpy --record=file.mp4 |
Do you have write permissions in Try this with |
|
Just tried this exe file, and when I try scrcpy --audio-output-buffer=10, it seems to fix the problem. Also I tried this exe file but opened it directly, and it was the same as above, slightly higher pitch, affected by EQ. When I use sndcpy, it seems to be playing the sound through VLC. As for read-only, the location where I installed the scrcpy seems to force read-only, trying to figure out how to fix that. |
I solved the read-only problem, and recorded a video Edit: I used the original v2.0 exe file to record the video, but from this result, probably the same for other exe files: the recorded video sounds okay, but the audio heard in real time is not so good. |
|
Just for information, do you experience the same "pitch issue" with |
|
I tried the numbers 10, 20, 50, 80, 100, with the following results: I also tested some numbers larger than 100, like 200, 500, 1000, but generally the situation is the same as 100. Also I tried to record the footage during the process, and I found that the recorded video had perfectly normal pitch in all cases. It seems that this is also a problem that exists only in real time audio. |
On which parameter? What was the value of the other parameter? I'm interested specifically with these two options simultaneously: |
I used "--audio-output-buffer" for all values when I tested. |
|
OK, so now that I have a bit more time (I quickly posted the previous comments from my phone), here are some more explanations about the issues you experience with the audio player (which do not impact capture/recording). The audio player buffers the decoded audio samples in the "audio buffer" (having a size determined by To play the sound, a (SDL) callback is called every 5ms to request 5ms of audio samples (which is 240 samples at 48000Hz). Scrcpy takes 5ms from its audio buffer to provide them to the system (this 5ms buffer is not under the control of scrcpy). The root cause of your problem is that some audio systems (platform/card/drivers) can't handle such low buffering (5ms), so instead of requesting 5ms every 5ms, they request 5ms every… 10ms, or even more. This is obviously problematic, it causes #3793 (refs libsdl-org/SDL#7450) and "the audio you hear (that) is severely broken and completely unlistenable". Therefore, I added an option
If for example your system requests 480 samples every 512 samples (10.7ms), then there is still a problem: audio samples are consumed a bit slower than they are received/produced. As explained in the release blog post, a compensation mechanism allows to adjust the stream to compensate for clock drift to maintain the target latency. Therefore, this mechanism will "speed up" a bit the input stream to match the consumption speed, hence the slightly higher pitch you get. It can probably be observed if you comment So in your case, you might need even more than 10ms of audio output buffer. You might try In theory, it is better to have the lowest possible value for this "audio output buffer" (but not lower that what your system supports):
If you increase the output buffer, you might need also to increase buffering in the internal scrcpy buffer (the "audio buffer") by the same amount (if the system requests samples less often, you need more buffering, so
In theory, with an "audio output buffer" sufficiently large (20ms is already very high), the SDL callback should not miss samples, so this should not cause compensation… unless this indirectly causes buffer underflow that needs to be compensated by playing the input stream faster. To confirm, try with
Yes, the API used to capture is different (playback capture), but it requires an app on the device and let applications to refuse to be captured.
The audio is raw (not encoded), and above all it does not attempt to keep the latency low (no compensation mechanism to maintain a target buffering level). To summarize: your system/card/drivers does not support low latency "audio output", and this has consequences that can be worked around by increasing latency (adding delay to both "audio output" and "audio output buffer"). |
|
Thank you for your detailed reply! |
It is not expected. Please post the output of
If you let it running for several minutes, does the latency increase? |
In my own experience, sndcpy has not had any major latency problems over long periods of time. Here is what happens after I use "scrcpy --audio-output=200 --audio-output-buffer=15 -Vverbose" as you said. D:\scrcpy-win64-v2.0>scrcpy --audio-output=200 --audio-output-buffer=15 -Vverbose |
Ouch, there is really a problem if you get this with an audio output buffer of 15ms. |
|
What I don't understand is that earlier you said:
So at 10ms, you don't get the problem, but at 15ms youget the problem? |
|
Sorry, maybe I didn't express myself correctly. |
But in the logs you posted, the audio is completely broken, since it includes many lines like this one:
and an insane level of buffering:
(11280 samples = 235ms!) It is clear that the audio output does not consume audio at the expected rate. |
|
I don't know much about software development, so I'm not sure to what extent you're referring to as "broken". I tried it again this time with "scrcpy --audio-output-buffer=100 -Vverbose", and this time there is no more "frequent delay" as mentioned above, although the pitch still increases for a few seconds. The logs look much more normal. Everything is back to the way it was at this time: #4055 (comment) |
That is not normal.
I think it's a software or driver issue at this point. I built a release from diffdiff --git a/app/src/audio_player.c b/app/src/audio_player.c
index 8f0ad7fba..2942de170 100644
--- a/app/src/audio_player.c
+++ b/app/src/audio_player.c
@@ -5,7 +5,7 @@
#include "util/log.h"
-#define SC_AUDIO_PLAYER_NDEBUG // comment to debug
+//#define SC_AUDIO_PLAYER_NDEBUG // comment to debug
/**
* Real-time audio player with configurable latencyPlease try:
For example with 20ms output buffer: |
I don't know why, but when I try to download this file, I get "403 Forbidden", can you please share this zip file in another way? |
|
Oops, wrong link. Fixed. |
|
Thank you for the kit. |
|
Follow up: Except for the first 5 seconds, the music sounded basically normal, without any problems. Follow-up again: |
|
Oh, there is indeed a difference between the rate at which samples are produced and consumed, but in your last logs, the culprit seems to be the producer (i.e. the device, it produces 960 samples when the consumer consumed 4×960 samples). (Btw, is it over USB or wifi?) I don't understand how recording could be perfect with that problem though. Before adding more logs, could you test without video and without audio encoding: (and without your equalizer, whatever it is) |
I just tried this, using the kit you provided, and the situation is basically the same as " #4055 (comment) ". |
|
OK, so let's add a log on the server side: diffdiff --git a/server/src/main/java/com/genymobile/scrcpy/AudioCapture.java b/server/src/main/java/com/genymobile/scrcpy/AudioCapture.java
index 7b20cce4f..46eeb1b27 100644
--- a/server/src/main/java/com/genymobile/scrcpy/AudioCapture.java
+++ b/server/src/main/java/com/genymobile/scrcpy/AudioCapture.java
@@ -167,6 +167,8 @@ public final class AudioCapture {
}
previousPts = pts;
+ Ln.i("=== [" + (System.nanoTime() / 1000) + "] pts=" + pts + ", " + (r / (CHANNELS * BYTES_PER_SAMPLE)) + " samples");
+
outBufferInfo.set(0, r, pts, 0);
return r;
}Here is a server binary (to be replaced in the release provided in #4055 (comment)):
|
I copied the file you provided and overwrote it, and tried this code again.
Then the logs started to grow rapidly, like below. |
|
OK, thank you for your tests. So the audio capture is correct. This is consistent with the fact that the recorded file is correct, but not consistent with the buffer underflows in #4055 (comment) (so it is caused by something else). I don't know what the problem is, but the behavior is clearly not correct in your case. Do you have access to another Android device (on the same computer), or another computer (to mirror the same Android device)? |
|
Let's log the actual rate on both sides of the audio buffer: diffdiff --git a/app/src/audio_player.c b/app/src/audio_player.c
index 8f0ad7fba..5eb914d05 100644
--- a/app/src/audio_player.c
+++ b/app/src/audio_player.c
@@ -5,7 +5,7 @@
#include "util/log.h"
-#define SC_AUDIO_PLAYER_NDEBUG // comment to debug
+//#define SC_AUDIO_PLAYER_NDEBUG // comment to debug
/**
* Real-time audio player with configurable latency
@@ -74,7 +74,8 @@ sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
uint32_t count = TO_SAMPLES(len);
#ifndef SC_AUDIO_PLAYER_NDEBUG
- LOGD("[Audio] SDL callback requests %" PRIu32 " samples", count);
+ LOGD("[Audio] [%" PRItick "] SDL callback requests %" PRIu32 " samples",
+ sc_tick_now(), count);
#endif
uint32_t buffered_samples = sc_audiobuf_can_read(&ap->buf);
@@ -91,8 +92,12 @@ sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
memset(stream, 0, len);
return;
}
+
+ ap->stats.start_play = sc_tick_now();
}
+ ap->stats.samples_played += count;
+
uint32_t read = MIN(buffered_samples, count);
if (read) {
sc_audiobuf_read(&ap->buf, stream, read);
@@ -116,6 +121,10 @@ sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
}
ap->played = true;
+
+ sc_tick duration = sc_tick_now() - ap->stats.start_play;
+ uint64_t freq = ap->stats.samples_played * SC_TICK_FREQ / duration;
+ LOGI("=== [aout] %" PRIu64 " samples played per second", freq);
}
static uint8_t *
@@ -141,6 +150,11 @@ sc_audio_player_frame_sink_push(struct sc_frame_sink *sink,
const AVFrame *frame) {
struct sc_audio_player *ap = DOWNCAST(sink);
+ if (!ap->stats.start_recv) {
+ ap->stats.start_recv = sc_tick_now();
+ }
+ ap->stats.samples_received += frame->nb_samples;
+
SwrContext *swr_ctx = ap->swr_ctx;
int64_t swr_delay = swr_get_delay(swr_ctx, ap->sample_rate);
@@ -315,6 +329,10 @@ sc_audio_player_frame_sink_push(struct sc_frame_sink *sink,
}
}
+ sc_tick duration = sc_tick_now() - ap->stats.start_recv;
+ uint64_t freq = ap->stats.samples_received * SC_TICK_FREQ / duration;
+ LOGI("=== [buffer] %" PRIu64 " samples received per second", freq);
+
return true;
}
@@ -425,6 +443,11 @@ sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
ap->underflow = 0;
ap->compensation = 0;
+ ap->stats.samples_received = 0;
+ ap->stats.samples_played = 0;
+ ap->stats.start_recv = 0;
+ ap->stats.start_play = 0;
+
// The thread calling open() is the thread calling push(), which fills the
// audio buffer consumed by the SDL audio thread.
ok = sc_thread_set_priority(SC_THREAD_PRIORITY_TIME_CRITICAL);
diff --git a/app/src/audio_player.h b/app/src/audio_player.h
index a03e9e35f..79424130d 100644
--- a/app/src/audio_player.h
+++ b/app/src/audio_player.h
@@ -77,6 +77,13 @@ struct sc_audio_player {
const struct sc_audio_player_callbacks *cbs;
void *cbs_userdata;
+
+ struct {
+ uint64_t samples_received;
+ uint64_t samples_played;
+ sc_tick start_recv;
+ sc_tick start_play;
+ } stats;
};
struct sc_audio_player_callbacks {
Please keep it open for 10~15 seconds (it will flood a lot). I suggest you redirect the output to a logfile and post that file here: scrcpy --audio-output-buffer=20 --audio-buffer=100 --no-video > logs.txt |
|
I don't have a second computer, but I do have a Pixel3, with Android 12, and I used it to do a comparison test as you said. |
|
Thank you, so the results look good to me. The only "problem" is that the audio playback takes time to start, so it starts way after the capture is started, so at the beginning it must "catch up" real time audio (by dropping samples, when So the only "problem" here is that the audio output takes time to start. Out of curiosity, is it better with another driver: #3799 (comment) :: in cmd
set SDL_AUDIODRIVER=directsound
scrcpy --audio-output-buffer=10
So in that case (apart from the first few seconds), you don't reproduce the initial issue where "the audio [you] hear is severely broken and completely unlistenable"? Can you reproduce it with To summarize what I have understood:
Am I correct? |
|
I tried it again a few times with your latest kit. Then when I tried values 5 and 10 in Directsound:
Out of curiosity, I also tried the If I don't switch to Directsound at the beginning:
Finally, regarding your summary, there is basically nothing wrong, I would like to add some more details about each value.
That's about it. Hope this helps you find something to fix the first 5 seconds. |
|
Hello, i seem to have the same issue but unlike OP, my phone and laptop are both fairly new. I am using an S21 Ultra with a HP Envy. scrcpy 2.1.1 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO: --> (usb) R5CR912GR3H device SM_G998B
/usr/local/share/scrcpy/scrcpy-server: 1 file pushed. 8.2 MB/s (56995 bytes in 0.007s)
[server] INFO: Device: [samsung] samsung SM-G998B (Android 13)
INFO: Recording started to mp4 file: aautput.mp4
INFO: Renderer: opengl
INFO: OpenGL version: 4.6 (Compatibility Profile) Mesa 22.2.5-0ubuntu0.1~22.04.3
INFO: Trilinear filtering enabled
INFO: Texture: 1080x2400
WARN: [FFmpeg] track 1: codec frame size is not set
^CINFO: Recording complete to mp4 file: aautput.mp4The audio is extremely choppy and it is basically completely unusable. I used to do it without the Happy to do any testing that is needed. |
|
Similar to OP's problem, realtime audio is a little high pitched recording is fine. Very noticable when using tcp but often times works normally on usb. Im using scrcpy for its wireless mirroring and touch support, so id rather if there is a solution to let me use tcp |
|
@rom1v I want to get in touch with you brother. Can you please allow me to do so? We are a non-profit organization, working for the visually impaired community. We want to use your SCRCPY in a specific environment, we need some assistance. Can you please help? |
Environment
Describe the bug
I upgraded to 2.0 but found that the audio quality from audio forwarding was extremely poor.
Although there are no errors reported, the audio I hear is severely broken and completely unlistenable.
I've tried all three encoders, adjusting the buffer, adjusting the bitrate, but nothing works.
And for some reason, the EQ app in my phone is affecting the sound. Although the sound quality is still very bad after turning off EQ.
In contrast, sndcpy works perfectly, and the sound is not affected by EQ. So I don't think the reason is my device.
Can I ask if anyone has the same situation? How can I fix it?
The text was updated successfully, but these errors were encountered: