HLS Live Streaming with DVR window and seek support added

[peddisri]

[Problem]
Exoplayer does not support HLS Live streaming with DVR window and seek. Issue #87

[Solution]

1. Support for available seek range added.
2. Support for seek within DVR window range added.

[Known Issues]

1. Seeking beyond the live window is handled as follows:
   a. If the seek position is less than lower bound, we start playback from the first
   available chunk data.
   b. If the seek position is greater than upper bound, we start giving data from the live
   edge, however, exoplayer skips frames till it gets the data corresponding
   to the seeked position
2. We assume either media sequence or program date time is always present in the media playlist.
   Playlists without media sequency or program date time is not handled. Behaviour is
   unpredictable.
3. Media playlist refresh time is aggresively fixed at half the target duration.
   We are yet to honor the spec that says if the last fetch did not change
   the playlist, try again after half the target duration, otherwise
   try after the target duration.
4. We assume that playlist always have timezone in program date time or
   never have timezone.
   If this assumption fails, live DVR functionality will be broken.

[Tests]

1. Manifests with either program date time or media sequence or both tested.
2. Manifests with variable segment duration is tested.
3. Adaptive bitrate Switching tested. Also tested the case where available
   range might be different across variants.

[perchrh]

Should this be TYPE_TEXT instead? It's about the textDataSource.

[perchrh]

Thanks for sharing your work, @peddisri!

How to test? I wanted to test this with my live streams, but ran into an issue where playback
starts at live head - 3 chunks (good), but the seekbar is at the beginning of the stream, stopping me from seeking backwards in the stream, for example to the beginning of the live window.

As a workaround, I tried moving the seekbar to the end and wait a little while before seeking to a position in the DVR window, but that doesn't seem to give consistent results - any position I seek to the first time seems to be considered the new start of the stream, in the sense that it seeks to chunk index 0 (HlsChunkSource#getLiveChunkMediaSequence's chunkIndex = 0) always.

Are additional UI changes necessary to test seeking, when using the ExoPlayer Demo app?

Btw: This PR would be easier to read and understand if it was split into multiple commits :)

[peddisri]

@perchrh The seekbar will always be at the start position because duration is UNKNOWN. However, you will see that the current time would indicate the live edge position and will monotonically increase.

You can use the FF and RW media buttons to seek. I use Amazon Fire TV for my tests and those buttons on the remote control work off the shelf. So with RW media button, you can seek backwards.

When you move the seekbar ,I am not sure what seek values would be passed when the duration is unknown. I've not tried to move the seekbar - instead use the RW and FF buttons only. Seekbar is not the right UI for Live streams anyways!

So please try with some remote or keyboard with Media Buttons. It is much easier that way.

[perchrh]

@peddisri Thanks for explaining. While a seekbar + current position and duration, like commonly used for VOD, is like you say not suited for live streams, a different style of seekbar can be a good solution for live streams. One use case is live TV, for channels with programs starting at specific times.

There are, I believe, two main ways of doing a live seekbar. In both cases the duration is not displayed, only the position. The position label can either be the offset from the live edge position, for example "-31:05", or (preferably) it can be the wall clock time associated with that position in the stream, for example "11:37:45". The wall clock time value can be calculated from the HLS' tag PROGRAM-DATE-TIME. If that tag is missing, an approximation can be made based on the time on the device when the live playlist was downloaded. This way, the seekbar can be used to, for example, find position "11:30", when a certain TV program starts.

I suggest this behavior for ExoPlayer HLS-live: to include a working seekbar, and to display "Live" in the duration label and the wall clock time, like described above, in the position label. The wall clock time associated with the current position can be calculated by determining the wall clock time when the stream starts (start of first chunk), and adding the elapsed time since the start.

[peddisri]

@perchrh The UI enhancements that you suggested makes sense - but can be implemented as as separate change.

[peddisri]

@ojw28 Did you get a chance to review this change?

[ojw28]

We have no plans to merge any DVR / seeking-in-live support for HLS until 2.x, where the architecture will look quite a bit different and should be more amenable to supporting this functionality.

[peddisri]

@ojw28 Thanks for the update! Do you have any time frame for 2.x?

[bricker]

@peddisri Thanks for doing this. I'm using it and it works well.

I'm trying to get access to the programDateTime attribute for the current chunk from within my app. Can you suggest a good way to do that?

[peddisri]

@bricker By "current chunk" you mean the one that is playing currently right? Unfortunately, there is no way for HlsChunkSource or HlsSampleSource to know which chunk is being played currently.
It only keeps track of the currently downloading chunk.

Also, I have only parsed the program Date time of the first chunk (not all the chunks), and I have stored this as part of the Media Playlist only (and not each segment) since, this was not required for HLS DVR use case. So, you would firstly need to parse the program date time of all the segments and and store as part of each segment. Minor changes would be required to HlsPlaylistParser.java and the Segment class for this.

If you need this information for the currently downloading chunk, the ideal place would be to send this as part of the onLoadCompleted event for Chunks. Unfortunately, this information is not available in HlsSampleSource.java and it would require some changes in TsChunk etc to store this info.

Alternatively, the other place where you could send out a new notification to app would be in HlsChunkSource.java in function getChunkOperation just before the last return statement. Here you have access to the segment variable that will now have the program date time field. (after the changes mentioned in second para above)

Example:
notifyxxxx(segment.programDateTime);
out.chunk = new TsChunk(dataSource, dataSpec, trigger, format, startTimeUs, endTimeUs,
chunkMediaSequence, segment.discontinuitySequenceNumber, extractorWrapper, encryptionKey,
encryptionIv);

[bricker]

Ahh great @peddisri — I will try that out.

[bricker]

@peddisri I have found a possible issue with this patch. The buffer start time gradually falls out of sync, and after about 20 seconds of playback (the approximate size of one segment), I begin seeing Discontinuity detected messages and the audio resets to the beginning of the last segment (it seems). I have tried looking for the cause but haven't found anything yet.

To be sure, I tested on the latest google/dev branch and this issue does not occur there. Only on this patch. I also merged the latest dev into this branch and the issue still occurs.

[bricker]

More details: The discontinuity issue seems to follow a pattern. In the following logs, "size" is the buffer size (in bytes) from inside AudioTrack/handleBuffer.

You can see that we're at 8192 bytes per buffer, and then we get the "OMXClient" message, size switches to 4096 and we experience a discontinuity. This happens 9/10 times I tested, but not every time.

04-30 17:57:45.992 4918-5010/com.org D/HLS: size: 8192
04-30 17:57:46.004 4918-5010/com.org D/HLS: size: 8192
04-30 17:57:46.022 4918-5663/com.org I/OMXClient: Using client-side OMX mux.
04-30 17:57:46.063 4918-5010/com.org D/HLS: size: 4096
04-30 17:57:46.063 4918-5010/com.org E/AudioTrack: Discontinuity detected [expected 3619851439, got 3599974161]
04-30 17:57:46.071 4918-5010/com.org D/HLS: size: 4096
04-30 17:57:46.075 4918-5010/com.org D/HLS: size: 4096
04-30 17:57:46.079 4918-5010/com.org D/HLS: size: 4096

[peddisri]

@bricker In my opinion this may not be due to this patch. Most likely the reason why the buffer size switches from 8192 to 4096 is because of change in channel count or sample rate in the stream. Assuming the audio content is AAC, in the logs, do you also see any log from SoftAAC2 decoder module - something like "reconfiguring...." at the time when the buffer size changes.. This log is a system log thrown by software AAC decoder in the device.

You may want to check if all the variants of the HLS stream that you are using to repro this issue are of same channel count and sample frequency. It is not advisable to use different audio properties because the audio codecs are not adaptive - it will result into re-creating of the decoder and could potentially cause a glitch at the time of switch.

Can you share the URL here or privately for me to repro this issue and investigate further.

[peddisri]

currentAvailableRangeIndex must be reset to -1 here otherwise when the user selects a specific track after starting the playback, it will throw out of array exception.

[oleynikd]

Any updates?

[oleynikd]

Dear @ojw28 what are plans for 2.x? Thanks.

[bene25]

@peddisri patch works with my stream. But there is nothing happens with seekbar. Could you help please. We need to implement DVR with ExoPlayer. Thanks.

[peddisri]

Concept of seekbar in Live DVR content is tricky and does not makes sense too. Ideally, if its a live content, it is advisable to not show the seek bar, instead just show current time increasing and a "Live" text.

[oleynikd]

Personally I think a seek bar is not the best solution for live. Have you guys seen how QuickTime handles this? It shows "30 sec back" button if there are enough chunks to go 30 sec back. And "back to live" button if there are chunks "in future". It is very simple in controlling and simple to understand.

[bene25]

Have you checked brightcove player - http://docs.brightcove.com/en/perform/mobile-sdks/brightcove-player-sdk-for-android/?
With help of seekbar you can go back about 30 minutes.

[peddisri]

@bene25 Conceptually, seek bar should have a cursor that progresses from left to right (or min position to max position). But in case of DVR window, the current position ideally should always be fixed, because the relative distance between current position and the end position should remain same, due to the sliding DVR window.
Hence I feel, showing seek bar to a end user is confusing.

[bricker]

YouTube's seek bar for live streams is a good example of why seek bars are useful for live content. Allowing the user to seek backwards to any point in a live broadcast, even while the stream is still occurring, is invaluable.

[ishantsagar]

@peddisri In the current fix when I press rewind(<< button), I reach to the same position from where the video started. Can I even go beyond that to a more previous state and not from where video started playing?

[peddisri]

@ishantsagar What is your DVR window range?