Seeking accuracy and error in version 2.7.0

[KKXLC]

still facing problem in seeking accurately in Exoplayer. we are facing this problem of inaccurate seeking when ever we are trying to seek to a time or play the file from a particular time.
say if we start playing a media file *.mp4 (<15mb) to a particular point say "x".
if we play a file giving the player position from were it has to play that media it seeks to that position and stat playing but inaccurately. to get error we did below steps:

we again seek to point "x1" at current time CT1.
so the player position will change as the time changes so it means at time (CT1+t) the player position should be at (x1+t).
but when we fetch the player position at (CT1+t) we are getting (x1+tt). so (tt-t = e), e is the error which ranges from 700 ms to 100 ms easily.
we are playing media at 1x player speed and content is fixed 24fps or 25 fps.
please help. how to remove this error and get at least 50 ms accuracy.

[KKXLC]

@andrewlewis its not typo but actually we see error ranging till 700 ms. yes CT 1 is current system time at which player has seeked to point x1 and ready to play.

and yes the stream has just audio in mp4 format.

previously also we were facing this problem then we spliced the big file of 100 minutes in small parts of 5 minutes so that file size becomes small and this lag is decreased . it had improvement but still error in AMPP(actual media player position) returns with error of 100s of ms when compared with IMPP (ideal media player position.: which is the position of the played it should have been ideally if no lag would have occurred.) to solve this we were trying to seek again and again after 1 sec hoping for correction but i believe that is like temporary lucky patch and not reliable. pls help. thanks

[ojw28]

I don't really understand what the problem is.

we again seek to point "x1" at current time CT1. so the player position will change as the time changes so it means at time (CT1+t) the player position should be at (x1+t).

This is incorrect. At time CT1 the player will start to buffer media from the newly specified seek position, possibly over the network depending on where your media is located. So playback wont start immediately, but will instead start some time later (CT1+bufferingTime). So at (CT1+t) you'd actually expect the playback position to be (x1+t-bufferingTime).

Is the problem that once playback does start, you see the position advancing slightly slower than you expect, causing a drift that gradually increases over time? If so then it's unclear why, but it's the underlying platform that's controlling the rate of output. At the ExoPlayer level we're just reporting the playback position in the media that's been reached, and I'm pretty sure we're doing this correctly. Also, if this is the issue, then why is it actually a problem for the app you're developing? What's the use case that you need to implement? If you're trying to synchronize something to the media then you should use the playback position that ExoPlayer reports (and if there is drift, poll the position frequently).

[KKXLC]

@ojw28 what you are saying is correct that at CT1+t the player should be at the x1+t-buffer time or say at CT1+t+buffer time the player would be at (x1+t).
but the problem is not that as we have already adjusted buffer time after monitoring the player state change and the buffer time varies every time you seek.
but what we are facing is that once we seek to (x1+t) position at (CT1+t) so we expect the player to seek at (x1+t) at (CT1+t+buffer time)
now at say time CT2=(CT1+t+buffer time + 5000 ms ) we expect the player to be at x1+t+5000ms : we call this as ideal player position (IPP). we fetch the current position of player @ (CT2=(CT1+t+buffer time + 5000 ms )) and the retrieved position is saved as APP (actual player position). so ideally APP should be equal to x1+t+5000ms (IPP) as the player is running linearly.
but we find there is difference between IPP and APP which could range from 10 , 20 ms to even 700 ms or even higher. on average of 100 seeks we found the data to be 200 ms + apart but huge variation.

secondly we understand that APP can have error of +/- 50 ms when we get it from the player ,

thirdly the player will take max 500 ms to stabilize after seeking ,

fourthly the player will seek to nearest frame and as we are playing 24fps /25fps content by this logic it should at least seek with an max error of 40-45 ms but its not the case as what we observed.

Use case: we are doing second screen/ multiple screen synchronization so we need to seek accurately at least with the accuracy of 100ms (max).

i hope that you must have got the problem . please help. thanks.

[ojw28]

I think the problem is being described in a way that's more complicated than is necessary, which is making it pretty hard to follow what you're actually observing. Does your media have audio? If so is there any audible discontinuity during playback? Are you sure the player doesn't stop to re-buffer? If audio sounds smooth then it's difficult to see how we'd get that far out of sync with the realtime clock. I'm pretty confident our reported positions accurately reflect where playback has reached in the media being played, and we've not had anyone else report this kind of issue despite ExoPlayer being widely used.

The above aside, at a high level, I don't think you're ever going to get the kind of synchronization you need for this use case with the approach you're trying to use. If you need to synchronize multiple screens then you really need to inject a properly synchronized (e.g. with NTP) into ExoPlayer, and use that to drive the playback. We don't support this use case directly, but I know people have done this successfully in the past by digging around in ExoPlayer's internal.

[KKXLC]

• yes the media has audio. its played from local file system.

• there is no discontinuity while playing . it plays in flow. we can hear the discontinuity if while playing we seek in the player: that is because the player buffers and state changes. one can hear a small click/silence.

• yes we land up that far and for reference i'm sharing the log in which we are printing actual player position (ampp) and ideal media player position (impp) and system current time in millis.
  

as you could see in the log the deviation is printed in milli sec and this is after seeking . this deviation could be 600 or 700 or 100 and it varies.

our synchronization dont require NTP clock synchronization as we have worked a way around that. we just require the player to seek to particular time stamp with minimal error. (max error 100ms)

[vikas3121]

@KKXLC did you get any solution in this issue??
Please share it if you any solution, as I am also facing a similar issue.

@ojw28 This is the actual issue which I also observed in ExoPlayer.
Here is my Problem description:

I have a time T1 at which I have to seek my my audio. Following is the code:

player.seekTo(T1);

When I have put a log in onSeekProcessed() and onPlayerStateChange() methods.

@OverRide
public void onSeekProcessed() {
Log.e( TAG, "On Seek Processed "+player.getCurrentPosition() ); // result is ==> 145624356
}
// After 1 Seconds I again did player.getCurrentPosition() and
//Result is something Shocking =====>145625163

Ideally after 1 sec = 1000 ms it should be at 145625356 however there is a clear difference of 193.

I executed same steps 4 5 times however, all the time getCurretPosition is giving different values.
Sometime this difference is 120 ms and sometime it is 134 ms.
I can understand that If EXO PLAYER works on frames then there should be an error of 41 ms or hardly 50 ms but not more than that as my video is 24fps.

NOTE All the media files are locally stored at the mobile phone. I am not streaming on network.

@ojw28 and @andrewlewis could you please put some light on this uncertain behavior of the player.
Sometime I also found that when we seek it eats almost 100 ms to play that audio.
So the time it is taking in buffering should be added to the seekPosition. However as this buffering time is also not certain so we are not sure how much time to be added exactly to the seekPosition in advance so after seek it can play from the exact position we want it to play.

Please suggest something in this regard.
Thanks in advance!!!

[andrewlewis]

Position discrepancies of less than about 130 milliseconds may be down to the audio track position not increasing perfectly linearly, tracked by #3841.

[tonihei]

@vikas3121 Just to emphasize it again. The difference of 193 ms you are seeing is most likely the time needed to rebuffer the media from the seek position. Can you make your measurements from onPlayerStateChanged to STATE_READY which is when the player finished buffering and starts playing?

[andrewlewis]

Yes. The audio track position non-linearity is only really relevant once you're measuring the point at which the player becomes ready correctly, and are compensating for any buffering that occurs (both in response to the seek and later on before you sample the position).

[ojw28]

I think you're just not properly synchronizing with when playback actually starts. Playback does not start instantly. onSeekProcessed isn't the point at which playback starts either. As @tonihei says, playback actually starting should be observed using onPlayerStateChanged.

[vikas3121]

@ojw28 I have performed several observations and following are stats:

I have used one mp4 file and log time are also saying something for the performance of this.

step 1: seeked player to say x1= 5000 ms and noted the current time as CT1 called first time.

step 2: re seeked player after 3sec at CT2 called to point x2=10000ms.

step 3:monitored player state: printed CT2@Buffering,CT2@ready in the second seek and calculate Buffer time as BT2.

03-01 20:46:35.450 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: __ STATE_BUFFERING __
03-01 20:46:35.450 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: Seeked to 5000 ms , CURRENT POSITION 5000 (getCurrentPosition())
03-01 20:46:35.755 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: __ STATE_READY __
03-01 20:46:35.755 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: Bufffered Time 305
03-01 20:46:35.755 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: CURRENT POSITION 5000 (getCurrentPosition())
03-01 20:46:38.394 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: Expected Player Position :7668
03-01 20:46:38.395 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: Actual Player Position :7363 (getCurrentPosition())
03-01 20:46:38.395 14712-14712/com.test.exoplayer E/ChunkAudioPlayer:
03-01 20:46:38.395 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: after 3 sec , seeked to 10000 ms
03-01 20:46:38.403 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: __ STATE_BUFFERING __
03-01 20:46:38.403 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: CURRENT POSITION 10000 (getCurrentPosition())
03-01 20:46:38.416 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: __ STATE_READY __
03-01 20:46:38.417 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: Bufffered Time 13
03-01 20:46:38.417 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: CURRENT POSITION 10000
03-01 20:46:39.401 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: Expected Player Position :11006
03-01 20:46:39.401 14712-14712/com.test.exoplayer E/ChunkAudioPlayer: Actual Player Position :10795 (getCurrentPosition())

At this position player should be at 11006 ms however it was at 10795 ms . Deviation received was of 205 milliseconds.

Similarly when we seeked it at 10000 ms after 3 seconds of first seek it should be at 8000 ms before seek however it was at 7668 milliseconds. It took 305 milliseconds as buffering time to be in state ready from the seek call.
Next time Buffered time was reduced to 13 ms.

Following is the code snippet for the same:

**@Override
public void onPlayerStateChanged(boolean playWhenReady, int playbackState) {

    if (playbackState == Player.STATE_BUFFERING){
        Log.e(TAG,"__  STATE_BUFFERING __");
        bufferingTime = System.currentTimeMillis();
        Log.e(TAG,"CURRENT POSITION "+player.getCurrentPosition());

    }

    if(playbackState == Player.STATE_READY){
        Log.e(TAG,"__  STATE_READY __");
        mStateChangeTimeDiff = System.currentTimeMillis() - bufferingTime;
        Log.e(TAG," Bufffered Time "+mStateChangeTimeDiff);
        Log.e(TAG,"CURRENT POSITION "+player.getCurrentPosition());

    }
}**



We are using following method to seek it with seek paramaters:

                **player.setSeekParameters(SeekParameters.NEXT_SYNC);
		player.seekTo(5000);**
				
We also tried **SeekParameters.EXACT**  , however there was no difference y using **next_sync** also. Result was almost same in both the cases.

[ojw28]

I'm able to reproduce similar results, although I can't get a deviation any higher than ~100ms. I have no idea how you're able to get such a high deviation, although it could be a result of your app performing a lot of work on its main thread (see below).

As previously stated, I think you're just not properly synchronizing with when playback actually starts. onPlayerStateChanged reporting STATE_READY is the closest you can get, and for most use cases (e.g. updating the state of playback controls) it's perfectly adequate, but it's not guaranteed to exactly coincide with audio starting to play out of the device, which is what it really means for playback to actually start, and is what the current media position reported by ExoPlayer is synchronized to.

There are multiple reasons why onPlayerStateChanged reporting STATE_READY may not exactly coincide with actual playback. Firstly, this event is posted for delivery on your application's main thread. If your application's main thread is busy handling lots of other work, for example UI layout or handling of user input, then delivery of the event will be delayed until that work has been done. Secondly, this event is posted when we call play on the underlying AudioTrack, but there's typically a small amount of output latency in the underlying platform that can slightly delay actual playback. What you should see is that once playback does start, if you periodically query the current position and the system clock, you'll see them advancing at exactly the same rate, give or take a few milliseconds of noise. It's just the initial synchronization that you're struggling with.

So I think in conclusion:

• Current position advances at the correct rate once playback starts.
• As I mentioned much further up, I don't think you're ever going to get the kind of synchronization you need for your use case with the approach you're trying to use, and I think you'd need to inject a properly synchronized clock somewhere much deeper into ExoPlayer. We don't support this use case directly, but I know people have done this successfully in the past by digging around in ExoPlayer's internals.
• You may be able to decrease the discrepancy by ensuring you're not doing a large amount of work on your application main thread, which may delay deliver of onPlayerStateChanged.