Automatically recover from temporary outages in live streams.

[peyoh]

Hello,
I'm using the latest commit from github.
We are using mpeg-dash.
Example MPD file you can get from here:
https://drive.google.com/file/d/0Bx_EoTGLxRt3Zks3VC16ZDU4Y2c/view?usp=sharing

Sometimes the video stops, and the only thing I can see is in the nginx access log.
The player try to reread the same audio files again and again.

How I can see what is wrong?

• - [11/Sep/2015:11:23:59 +0300] "GET /dash/bnt1/bnt1.mpd HTTP/1.1" 304 0 "http://-:81/dash/shaka-player/" "Mozilla/5.0 (Linux; Android 5.0.2; Lenovo K920 Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.84 Mobile Safari/537.36" "-"
• - [11/Sep/2015:11:23:59 +0300] "GET /dash/bnt1/bnt1_edash-audio-288.m4a?_=1441959842344 HTTP/1.1" 200 162912 "http://-:81/dash/shaka-player/" "Mozilla/5.0 (Linux; Android 5.0.2; Lenovo K920 Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.84 Mobile Safari/537.36" "-"
• - [11/Sep/2015:11:23:59 +0300] "GET /dash/bnt1/bnt1_edash-audio-289.m4a?_=1441959842451 HTTP/1.1" 200 162885 "http://-:81/dash/shaka-player/" "Mozilla/5.0 (Linux; Android 5.0.2; Lenovo K920 Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.84 Mobile Safari/537.36" "-"
• - [11/Sep/2015:11:23:59 +0300] "GET /dash/bnt1/bnt1_edash-audio-290.m4a?_=1441959842492 HTTP/1.1" 200 162917 "http://-:81/dash/shaka-player/" "Mozilla/5.0 (Linux; Android 5.0.2; Lenovo K920 Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.84 Mobile Safari/537.36" "-"
• - [11/Sep/2015:11:24:04 +0300] "GET /dash/bnt1/bnt1.mpd HTTP/1.1" 200 2878 "http://-:81/dash/shaka-player/" "Mozilla/5.0 (Linux; Android 5.0.2; Lenovo K920 Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.84 Mobile Safari/537.36" "-"
• - [11/Sep/2015:11:24:04 +0300] "GET /dash/bnt1/bnt1_edash-audio-288.m4a?_=1441959847371 HTTP/1.1" 200 162912 "http://-:81/dash/shaka-player/" "Mozilla/5.0 (Linux; Android 5.0.2; Lenovo K920 Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.84 Mobile Safari/537.36" "-"
• - [11/Sep/2015:11:24:04 +0300] "GET /dash/bnt1/bnt1_edash-audio-289.m4a?_=1441959847410 HTTP/1.1" 200 162885 "http://-:81/dash/shaka-player/" "Mozilla/5.0 (Linux; Android 5.0.2; Lenovo K920 Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.84 Mobile Safari/537.36" "-"
• - [11/Sep/2015:11:24:04 +0300] "GET /dash/bnt1/bnt1_edash-audio-290.m4a?_=1441959847445 HTTP/1.1" 200 162917 "http://-:81/dash/shaka-player/" "Mozilla/5.0 (Linux; Android 5.0.2; Lenovo K920 Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.84 Mobile Safari/537.36" "-"

BRS/
Peyo

[tdrews]

You can enable logging in the demo application by adding the query parameters ?debug or ?v to the URL, e.g., http://localhost/shaka/?debug

Alternatively you can call shaka.log.setLevel(shaka.log.Level.DEBUG) or shaka.log.setLevel(shaka.log.Level.V1) in your own application (in uncompiled mode).

[peyoh]

Hello,
now with the console I'm able to see the error. Is possible the player to recover from this error? Or how the player can continue to play even with some "sqeezes"?
Please check the console log attached:
https://drive.google.com/file/d/0Bx_EoTGLxRt3QjRSWGp6ejBvdVk/view?usp=sharing

BTW,
I'm using edash_packager.

BRS/
Peyo

[tdrews]

After reviewing the logs, I believe the content has "large" gaps (> ~1 second) in it, which is causing the player to stall. For live streams, these gaps may occur if the encoder & packager cannot keep up. Are you processing multiple streams in parallel?

(As an aside: the warning pertaining to @availabilityStartTime is generally harmless; however, you may be able to eliminate it by passing -availability_time_offset 0 to edash-packager.)

[peyoh]

Yes. There are some gaps in the input stream.
Unfortunately this is frequent situation when processing DVB-S stream. For example bad weather, clouds, rain. Some picture/audio breaks are not problem, but the player is unable to recover from the error and the user is forced to stop and start playing the channel again which can cause bad user experience.
If I can tell to the player to jump on the next segment (probably good), this will smash the picture(of course), but will no stop the service.

BRS/
Peyo

[tdrews]

We did not consider frequent gaps within a stream as a normal use case. We'll have to investigate further to determine if/when/how we want to address this issue.

As a workaround you may be able to add something like video.currentTime = video.buffered.start(video.buffered.length - 1); above https://github.com/google/shaka-player/blob/c45a3c273f8b285c07e095da1de8e55adbb3b181/lib/media/stream.js#L509 (although I have not tested this).

[peyoh]

Hello,
I've tried your suggestion, but without success.
Unfortunately I'm not javascript developer, so I will leave the future investigations on more skilled developers.
BRS/
Peyo

[peyoh]

Hello,
actually this is not so rare issue . This issue can be replicated with a very small gaps and only in Audio even the video is untouched.
This situation I've replicated in my lab. The video/audio transcoder just put a small gap (for example 283 bytes) in audio, when the input audio stream cannot be decoded ( missing header, PES mismatch etc.) . The video is not affected. But even then the issue is
visible and the player stops.

BRS

[joeyparrish]

@peyoh, can you provide us a sample of content that reproduces this situation for you? If you can't share it publicly, please feel free to send me a private email. That would really help us improve the resiliency of the system. Thanks!

[peyoh]

@joeyparrish, I don't know your private email. Where I can find it?
Once I have it, I will send you a link. From this link you can also reproduce #288

[peyoh]

@joeyparrish, I've just send you a email with URLs

[peyoh]

Hello,
I've noticed, that every time the player stops playing, this message appears in the debug console:

SBM audio/mp4: multiple buffered ranges detected: Either the content has gaps in it, the content's segments are not aligned across bitrates, or the browser has evicted the middle of the buffer. source_buffer_manager.js

How I can catch this event from the player? Because if I can catch it, I can try to move 1 sec forward.

BRS

[joeyparrish]

There is no event to catch. Shaka v1 assumes that there will only be a single buffered range, and has assertions to this effect. What you see are failed assertions. In Shaka v2, we no longer make this assumption, so this message will go away.

If you wish, you can listen for the 'bufferingStart' event from the Player. This event fires whenever the player enters a buffering state. When this happens, you can check to see if there's a buffered range which starts ahead of the playhead (video.currentTime), and if so, seek to it.

[joeyparrish]

Please consider this issue merged with #555. If you are following this, please go to #555 and subscribe. Future updates will appear there.