[BUG] recording segments timestamp drift over time

[SijmenHuizenga]

Describe the bug
The timestamp of .ts segments drifts away from real time.

We have a participant with a camera. The camera points at a clock. We expect the timestamps in the .ts segments to align with the time on the clock. When the egress starts, the timestamps align. Over time the timestamps drift apart.

This is a timeline of egresses EG_QdZtWe5x33u4 in cloud project p_2u70izoj75h:

Time into the recording (wall clock)	File	File timestamp (UTC)	Wall clock on first frame of recording (UTC+1)	Diff
0	testbed_20231108104238558.ts	10:42:38.558	11:42:37.28	~ 1 second
2647 seconds	testbed_20231108112632558.ts	11:26:32.558	12:26:44.56	~12 seconds
6253 seconds	testbed_20231108122620558.ts	12:26:20.558	13:26:50.59	~20 seconds

Files:
drift.zip

We see here that after a recording of 1 hour and 44 minutes, there is a 20 second drift. We will regularly be doing recordings of 24 hours. Interpolating the drift, that would mean we might end up with more than 4 minutes drift.

I used this command to extract the first frame of the segment: ffmpeg -i testbed_20231108122620558.ts -vf "select=eq(n\,0)" -vframes 1 out.png

Why is this a problem?

We need to show recordings of exact times. For example: users request to see what happened at 12:26:20. While this bug exists we are unable to find the correct recording.

Egress Version
Using livekit cloud. Not sure which version is deployed over there.

Additional context
Previously we've seen weird timing behavior when the stream metadata was misconfigured. This has since been resolved, so i don't think it's related. But who knows 🤷

[SijmenHuizenga]

Okay, i might have found a workaround...

We are using the livekit go sdk to stream. In there, we pass the frame duration to livekit like so:

track, err := lksdk.NewLocalReaderTrack(in, mime,
	lksdk.ReaderTrackWithFrameDuration(time.Duration(1000/track.fps)*time.Millisecond),
)

this camera is 15 fps, so the value sent to ReaderTrackWithFrameDuration would be 66 ms. This was a rounded value as the true value is 66.666666666.... I figured, maybe livekit uses this value to interpolate time, so i might as well try reducing the rounding. So i changed the code to:

lksdk.ReaderTrackWithFrameDuration(time.Duration(1000000000/track.fps)),

making the value 66666666 nanoseconds. It seems that this reduces the drift significantly to less than a second over 12 hours. That's acceptable to me.

For now my problem is resolved, but I still believe the behavior seen here is unexpected. Documentation on how the timestamps of the recorded videos are created would be extremely helpful.

[frostbyte73]

when streaming with the go sdk, the frame duration is the ultimate source of truth for timing information. It might be better if the sdk accepted a framerate and handled frame durations on its own