Technical question about audio sync

[ilwoonam75]

Hello.

First of all, thank you for your great project.

We are currently working on a video player project, and trying to use this project as a baseline.

I understand the video sync method with pts, but I don't understand how you synchronize audio with pts.

Can you explain how to sync audio?

The code block I don't understand is

if (decoder.hasAudio)
{
// Audio Sync
if (Math.Abs((audioDelayTicks - delayTicks) / 10000) > 220)
{
if (curTicks + audioExternalDelay < decoder.aStreamInfo.startTimeTicks || curTicks + audioExternalDelay > decoder.aStreamInfo.durationTicks)
audioDelayTicks = -10000000; // Force Resync Later (Audio Paused)
else if (ResynchAudio(curTicks - audioExternalDelay)) audioDelayTicks = delayTicks;
}
// Audio Frames [Callback]
if ( aFrames.Count > 0) SendAudioFrames();
}

[SuRGeoNix]

Hey, I'm glad you find useful :)

The audio sync is the most difficult part! I'm doing it by syncing audio samples with the video frames. At the beginning of Screamer you can see the following code:

// Audio
audioFlowTicks = DateTime.UtcNow.Ticks;
long audioDelayTicks = delayTicks;
if (decoder.hasAudio)
	if (vFrame.timestamp + audioExternalDelay < decoder.aStreamInfo.startTimeTicks || vFrame.timestamp + audioExternalDelay > decoder.aStreamInfo.durationTicks) 
		audioDelayTicks = -10000000; // Force Resync Later (Audio Paused)

> 	else ResynchAudio(vFrame.timestamp - audioExternalDelay, true);

After I make sure that Audio/Video queues are filled with the right timestamps and synced I keep track of audioDelayTicks/delayTicks. Those (delayTicks) are actually how much "off time" are currently the video frames and when they are much enough we reset them to the the current timestamp. But when the video frames are going off we don't reset audio directly (audioDelayTicks), in that way we would have to do it all the time. So when the distance is more than 220ms it will do the re-sync.

In that way you make sure that your video frames and your audio samples will be screamed almost at the same "allowed" time frame.

After that I use audioFlowBytes/ audioBytesPerSecond which are configured based on the audio configuration / sample ratio etc...

audioBytesPerSecond =(int)( _RATE * (_BITS / 8.0) * _CHANNELS);

to make sure that I will send the flow in the same rate that the audio player requires to keep the audio queue low (which means memory low). But even after all that, I still had a problem with the audio player (naudio). Each audio player has a different way that it works :) It means that NAudio requires some time/bytes to load its buffer before play. That's why you will see the comment...

const int NAUDIO_DELAY_MS = -330; // Probably waits to fill it's buffer? had to use this for sync https://www.youtube.com/watch?v=ucZl6vQ_8Uo

Finally, there are more thinks that you could add to make it better. For example, you could sync during silence. So the listeners will not notice the difference.

Hope that helped, but if not you could check more details about audio sync here:

Synchronizing the Audio

[ilwoonam75]

Thank you very much for your answer.

[SuRGeoNix]

No problem dude. You made me look back again the code and I found a couple of issues, check out my latest commit for details!

[ilwoonam75]

Let me check your new commit.
Thank you again.