Audio desynchronisation

[p10tyr]

System/Software Setup

• LiveVisionKit Version: 1.2.2
• OBS Studio Version: 29.1,3 64bit (Runs in admin mode)
• OS: Win11 21H2
• GPU: RTX 3060 Ti Founders
• CPU: Ryzen 5 5600X

OBS Log:: A link to the OBS log corresponding with the issue. This can be done through OBS by going to - Help -> Log Files -> Upload Current Log File

07:04:54.071: Last log entry repeated for 2804 more lines
07:04:54.071: adding 384 milliseconds of audio buffering, total audio buffering is now 384 milliseconds (source: SRT DIRECT - STABILISED)
07:04:54.071:

...

: Last log entry repeated for 2 more lines
07:10:20.325: MP: av_read_frame failed: I/O error (-5)
07:10:20.345: Max audio buffering reached!
07:10:20.345: adding 576 milliseconds of audio buffering, total audio buffering is now 960 milliseconds (source: SRT DIRECT - STABILISED)
07:10:20.345: 
07:10:20.365: Source SRT DIRECT - STABILISED audio is lagging (over by 1325.09 ms) at max audio buffering. Restarting source audio.
07:10:20.456: [Media Source 'SRT DIRECT - STABILISED']: Disconnected. Reconnecting...
redacted
07:10:32.170: [Media Source 'SRT DIRECT - STABILISED']: Reconnected.
07:10:33.576: [LiveVisionKit] '(LVK) Video Stabiliser' frame queue is outdated, restarting...
07:10:34.013: [LiveVisionKit] '(LVK) Video Stabiliser' released 11 frames.

...

07:30:52.176: Last log entry repeated for 91 more lines
07:30:52.176: Source SRT DIRECT - STABILISED audio is lagging (over by 1944.82 ms) at max audio buffering. Restarting source audio.
07:30:52.860: [LiveVisionKit] '(LVK) Video Stabiliser' frame queue is outdated, restarting...
07:30:53.243: [LiveVisionKit] '(LVK) Video Stabiliser' released 11 frames.

...

08:10:28.036: MP: av_read_frame failed: I/O error (-5)
08:10:28.075: [Media Source 'SRT DIRECT - STABILISED']: Disconnected. Reconnecting...
redacted
08:10:39.526: [Media Source 'SRT DIRECT - STABILISED']: Reconnected.
08:10:40.945: [LiveVisionKit] '(LVK) Video Stabiliser' frame queue is outdated, restarting...
08:10:41.387: [LiveVisionKit] '(LVK) Video Stabiliser' released 11 frames.

Issue Description
This is going to be caused by us live streaming from a location with bad internet. We are using a Solo Pro with SRT direct into OBS.
After turning off 3 filters, stabilise, sharpen and debuff the issue stopped.

Here is a clip from the live stream were presenter sounds like a dalek.

https://youtube.com/clip/UgkxhJgqhTHja_UaxBdJCdleiwZVRRsYZFUq

The only way to solve the issue is to stop the stream from the solo and start it again.. so everything restarts and then its fine.

I have no other filters just an overlay and then send it to YouTube - so turned the 3 filters off but I suspect it is the stabiliser filter that is trying to offsett the audio and then gets into some kind of broken state.

Expected Behavior
No audio jitter when SRT connection is bad, probably a lot of retransmit fails and causes audio packets to be lost.

Reproduction Steps
Pffff... donno - we been having this all weekend streaming 3 days over 7 hours each. we had it once 1 day, ,4 times another day and never on another. Just seems to be dependant on connection and maybe SRT retransmit failures. So you could maybe use the tool on SRT cookbook to force the network for packet loss.. its a linux command that does random packet losses. That seems to be the key in the problem here

Supporting Evidence
https://youtube.com/clip/UgkxhJgqhTHja_UaxBdJCdleiwZVRRsYZFUq

Appendix
OBS is stable. Runs in Admin mode. CPU is OK. System has more than enough resource, storage, ram. etc.

[p10tyr]

One thing that occurred to me is that sometimes we have a camera with 60fps and sometimes 30fps.. But OBS is always set to 60FPS.. maybe this is the cause of the synchronisation problem after some lag issues.

I will try next time to set OBS and Camera the same FPS and turn on stabiliser. See how it goes.

[Crowsinc]

Hi, thanks for the detailed issue write-up. Audio synchronization is handled by internally OBS. The stabilizer only affects it by adding a frame delay, which is reset whenever the filter gets a frame that's over a second old (e.g. when the stream lags). This is how the official Video Delay (Async) OBS filter works too.

There's definitely something going wrong whenever delay is reset due to the connection lag, which ill have to investigate. One would think that if there is a de-sync occurring within the frame queue inside the stabilizer, it should completely clear itself once all the de-synced frames have made their way out and are replaced by fresh frames from the Media Source. Which makes me think it may be a weird interaction with OBS audio buffering, since you mentioned the problem doesn't resolve itself?

Regardless, the whole delay resetting thing has been completely removed in the next update to avoid any potential issues like this.

[LonleyViper]

@Crowsinc I've also experienced the same thing. Around 3 hours into streaming, obs will not be able to maintain 60fps. The audio sync will deterioriate to a seriously bad state.

Turning off (LVK) Video Stabiliser resolves this.

I'm running a dedicated rig with a 2060 Super for encoding. From what I can tell, its a memory leak causing encoder overload. Restarting OBS fixes the issue.

Log file attached - hope this helps!

2023-07-21 19-57-03.txt

[LonleyViper]

Quick update to say my audio desync was coming from an issue with StreamFX using certain shaders/filters that caused issues even when switched off!

[Crowsinc]

Thanks for the update! Do you know which StreamFX filters were causing the issues?

[p10tyr]

Just and update. It is not LiveVisionKit that caused the dalek audio crackling.. it does exasperate it though but the root cause problem seems to be a SRT synchronisation problem when the stream data falls to 0 (4g outage) and then adaptively starts to come back.. the audio is messed up and we have to stop the input and start it.

On this instance I had no filters added to the input stream as I was monitoring for this issue and it happened today with 0 filters. Standard basic OBS.

[p10tyr]

Although just having the plugin in the plugin directory could affect it even if its not loaded on any scenes? I probably need to disregard last post.. remove the code and make it truly blank OBS and recheck.

[LonleyViper]

Thanks for the update! Do you know which StreamFX filters were causing the issues?

I think it was shake!

[Crowsinc]

Some fixes were introduced in the latest update. I have been unable to verify their effectiveness, however, I'm closing the ticket for now as the code-base has changed almost completely since this issue was raised.