[2.46][GST][MSE] Skip playback state update while EOS #1568
Conversation
Playback state change notification syncs Media Client (HTMLMediaElement)
state with current player state:
commit fb0730e:
The platform backend may change state, for example as a result
of an external plugin controlling the backend, so we need to
react to this situation by syncing up the WebCore state with the
platform backend.
We call playInternal()/pauseInternal() depending on the backend
state, to trigger the corresponding DOM events to match the state.
Sending that notification on EOS condition may effectively pause the pipeline
without user request becasue the GST player reports it is paused m_isEndReached is set.
|
Here is a test case for that https://asurdej-comcast.github.io/tr/tests/RDKTV-38576_mse_seek_at_eos.html The flow is: With current impl the playback gets stuck in paused state and never restarts |
asurdej-comcast
changed the title
|
Here is a sequence of events from WebKit perspective that leads to video stall:
My proposition fixes the bold bullet, While m_isEndReached is set, it doesn't send playbackStateChanged() notification to HTMLMediaElement so it will never accept the paused state and won't force it later on MediaPlayer |
|
I'm having a look at this next Monday Oct 6th. |
|
I'm still trying to understand all the steps, but I wonder (to myself) if, instead of ignoring the m_isEndReached flag as a sort of hacky workaround, there's any way of resetting the flag ahead of time (even before the seek starts to be processed asynchronously), so that it's false already at the right time. I'm trying to analyze if this is possible, for instance by using willSeekToTarget(), which sets m_pendingSeekTime. I still haven't checked if that call happens soon enough, tho. I'll continue analyzing it. |
|
I'm having a hard time to get your test case to work properly (at least on a Raspberry Pi, wpe-2.46), even after having mirrored it on a local server and added extra logs. First, I needed to increase the timeouts (the data load was slow for some reason). Then, I'm having spureous "can't push buffer because the pad is on EOS" failures in one of the WebKitMediaSrc pads. All this applying only your suggested fix, instead of mine. I think I only saw it working (green) one time after multiple tries. I'm still debugging what happens. |
|
@eocanha Thanks for looking into this. Disney plus uses the same MediaSource object and the same SourceBuffers to play both main content and advertisments. That is a bit tricky as MediaSource isn't accepting more data after endOfStream() so it has to be reopened somehow, I use timestampOffset of SourceBuffer that reopens MediaSouce. Resetting m_isEndReached, e.g. in MediaPlayerPrivate::play() would help but the GST is still at eos position so there are no more data to play so it will be reported by the sinks again. Another "problem" could be the ready state, that inside MediaSource it is calculated based on the new position (current position is taken from m_pendingSeekTime set from willSeekToTarget()) and propagated to player while the player is still at the old position |
|
I think I've solved the problem using a different approach that I would say that makes more sense: On the WPE side, I've cleared the But still, when I did that, WebKitMediaSourceGStreamer would error out because the pad was on EOS. I debugged that on the GStreamer 1.18 side and realized that GstDecodebin3 was doing non-obvious trickery with the GstMultiqueue srcpad EOS events. It would accumulate (memorize) EOS emitted by multiqueue until all the streams have emitted EOS. Then it would forward all those EOS at once to all the pads. This caused a puzzling behaviour where video buffers would be pushed downstream normally and, at some point, without apparent reason (when the audio pad received its EOS and triggered a chain reaction also on the video pad), start to be rejected because the pad was on EOS. I solved this by resetting the memorized I've cleaned them up and submitted them (as part of experimental branches) as WebPlatformForEmbedded/buildroot@ac1d6ff and 15b9b55. The GStreamer code has changed in 1.20, but I'm still going to try to see if the bug is reproducible upstream and it makes sense to upstream the WPE part of the patch at least, and/or debug the behaviour on latest GStreamer. |
…flush-stop Without this change, a video would be played till the end (so the video stream would EOS and the slot be amarked as such), then seeked back (but still be EOS), and when the audio pad gets EOS (because of its own circumstances that would trigger a chain reaction that would forward all the accumulated EOS to all the multiqueue src pads, causing all the video buffers to fail being pushed from that moment on. This change resets the flag on flush, preventing the issue. The slots won't still be considered drained after the flush and buffer pushing won't be interrupted. See: WebPlatformForEmbedded/WPEWebKit#1568 (comment)
When a video has reached End Of Stream and a (backwards) seek is done, the media player won't ever complete the seek because the EOS condition is still in place and no buffers are ever going to be pushed. A successful seek completion would reset the EOS condition, but that won't happen since the seek won't even start. This change resets m_isEndReached as soon as possible (on willSeekToTarget()), so the seek can happen. See: #1568 (comment)
|
Thanks @eocanha I've tried your Webkit patch alone and it seems to fix the problem, but... I don't see any gst errors connected with pushing data buffers after EOS. I'm aware that eos handling in gst is not trivial, we had some problems connected to that in the past. Not sure how about this particular problem, if we have any custom patch on top of GST or just gst version is different, but I dont see that error on our devices. Webkit just pushes data after seek/flush without any issue. In my case the WebKit patch alone does the job but I have some concerns. So it resets EOS flag in MediaPlayer while requesting a seek after initial end of stream (willSeekToTarget). Next we have a play request that will put the pipeline in playing state at the old position (seek hasn't been applied yet). We don't have any data to process at this point because the player is still at the end. So gstreamer pipeline will report EOS/didEnd() again, at the very first convenience, setting m_isEosReached=true again. The patch works because mediaPlayerPlaybackStateChanged() is called before that happens so HTMLMediaElement sees that the MediaPlayer is playing (m_isEndReached is false). Otherwise it looks good and the original problem is gone |
3 participants
Playback state change notification syncs Media Client (HTMLMediaElement) state with current player state:
commit fb0730e:
Sending that notification on EOS condition may effectively pause the pipeline without user request becasue the GST player reports it is paused m_isEndReached is set.
PS. We already had a bunch of problems with that playback state change notification and the way of how HTMLMediaElement handles mediaPlayerPlaybackStateChanged(). In 2.46 that part was added because of sleep disable while it was completely missing in 2.38. Maybe we should handle sleep disable separately and skip that mediaPlayerPlaybackStateChanged() at all
46455f0