Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Switching audio/subtitle track in HTTP/MPEG-TS stream restarts playback at the beginning #2956

Closed
goffioul opened this issue Jun 15, 2017 · 9 comments
Closed

Switching audio/subtitle track in HTTP/MPEG-TS stream restarts playback at the beginning #2956

goffioul opened this issue Jun 15, 2017 · 9 comments
Assignees
@ojw28
Labels
question

Comments

@goffioul
Copy link
Contributor

Using the demo app, I'm playing a MPEG-TS stream through HTTP. The stream contains multiple audio and subtitle tracks. Whenever I switch the audio/subtitle track in the demo app, the playback restarts at the beginning.
@ojw28
Copy link
Contributor

ojw28 commented Jun 15, 2017

This is because changing tracks currently triggers a seek, and MPEG-TS is not seekable. We are considering avoiding seeking on track selection, at the cost of using more memory, but right now this is working as intended.

@ojw28 ojw28 self-assigned this Jun 15, 2017
@ojw28 ojw28 added the question label Jun 15, 2017
@goffioul
Copy link
Contributor Author

Many other player I know of can seek within MPEG-TS. I understand it's imperfect, but still better than nothing. Anyway, thanks for the clarification. I hope you guys will actually implement smoother track switching.

ojw28 added a commit that referenced this issue Jun 26, 2017
@ojw28
Move ExtractorMediaPeriod to new SampleQueue methods
1b71e3b 
This change allows you to enable/disable tracks within which
all samples are key-frames without any re-buffering (e.g. audio,
text and metadata). This effectively reverts V2 back to the
behavior in V1, only this time we're doing it properly. []ly
disabling/enabling, or disabling/enabling whilst paused, no longer
cause samples to get "lost" between the source and renderers.

Note it also becomes really easy to support a few other things,
although support is not exposed in this change:

- Enable/disable video tracks without any re-buffering, by
  changing the toKeyframe argument passed to discardTo to true.
- Retain media in the buffer for some time after it's been played
  (e.g. to support a single back-5s-seek efficiently), by
  subtracting the desired back-buffer time from the value that's
  passed to discardTo.

Issue: #2956
Issue: #2926

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160128586
@ojw28
Copy link
Contributor

ojw28 commented Jun 26, 2017

This is fixed in dev-v2.

@ojw28 ojw28 closed this as completed Jun 26, 2017
@goffioul
Copy link
Contributor Author

I've tried the demo app from branch dev-v2, playing a MPEG-TS stream through plain HTTP, but I get an exception. Here are the logs, using this sample stream https://drive.google.com/open?id=0BwljeX6541LubUVmbGZad0QzcWM

06-26 17:57:02.792 14254 14254 I DefaultRenderersFactory: Loaded LibvpxVideoRenderer.
06-26 17:57:02.793 14254 14254 I DefaultRenderersFactory: Loaded LibopusAudioRenderer.
06-26 17:57:02.794 14254 14254 I DefaultRenderersFactory: Loaded LibflacAudioRenderer.
06-26 17:57:02.794 14254 14254 I DefaultRenderersFactory: Loaded FfmpegAudioRenderer.
06-26 17:57:02.794 14254 14254 I ExoPlayerImpl: Init ExoPlayerLib/2.4.2 [ls1853, ls1853, unknown, 25]
06-26 17:57:02.798 14254 14254 D EventLogger: state [0.01, true, I]
06-26 17:57:02.807 14254 14254 D EventLogger: state [0.02, true, B]
06-26 17:57:02.808 14254 14254 D EventLogger: sourceInfo [periodCount=1, windowCount=1
06-26 17:57:02.808 14254 14254 D EventLogger:   period [?]
06-26 17:57:02.808 14254 14254 D EventLogger:   window [?, false, false]
06-26 17:57:02.808 14254 14254 D EventLogger: ]
06-26 17:57:02.808 14254 14254 D EventLogger: loading [true]
06-26 17:57:02.867  1708  1729 I ActivityManager: Displayed com.google.android.exoplayer2.demo/.PlayerActivity: +123ms
06-26 17:57:02.884 14254 14254 D EventLogger: sourceInfo [periodCount=1, windowCount=1
06-26 17:57:02.884 14254 14254 D EventLogger:   period [?]
06-26 17:57:02.884 14254 14254 D EventLogger:   window [?, false, false]
06-26 17:57:02.884 14254 14254 D EventLogger: ]
06-26 17:57:02.885 14254 14297 E ExoPlayerImplInternal: Internal runtime error.
06-26 17:57:02.885 14254 14297 E ExoPlayerImplInternal: java.lang.ArrayIndexOutOfBoundsException: length=3; index=5
06-26 17:57:02.885 14254 14297 E ExoPlayerImplInternal: 	at com.google.android.exoplayer2.source.ExtractorMediaPeriod.selectTracks(ExtractorMediaPeriod.java:220)
06-26 17:57:02.885 14254 14297 E ExoPlayerImplInternal: 	at com.google.android.exoplayer2.ExoPlayerImplInternal$MediaPeriodHolder.updatePeriodTrackSelection(ExoPlayerImplInternal.java:1597)
06-26 17:57:02.885 14254 14297 E ExoPlayerImplInternal: 	at com.google.android.exoplayer2.ExoPlayerImplInternal$MediaPeriodHolder.updatePeriodTrackSelection(ExoPlayerImplInternal.java:1584)
06-26 17:57:02.885 14254 14297 E ExoPlayerImplInternal: 	at com.google.android.exoplayer2.ExoPlayerImplInternal$MediaPeriodHolder.handlePrepared(ExoPlayerImplInternal.java:1570)
06-26 17:57:02.885 14254 14297 E ExoPlayerImplInternal: 	at com.google.android.exoplayer2.ExoPlayerImplInternal.handlePeriodPrepared(ExoPlayerImplInternal.java:1377)
06-26 17:57:02.885 14254 14297 E ExoPlayerImplInternal: 	at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:344)
06-26 17:57:02.885 14254 14297 E ExoPlayerImplInternal: 	at android.os.Handler.dispatchMessage(Handler.java:98)
06-26 17:57:02.885 14254 14297 E ExoPlayerImplInternal: 	at android.os.Looper.loop(Looper.java:154)
06-26 17:57:02.885 14254 14297 E ExoPlayerImplInternal: 	at android.os.HandlerThread.run(HandlerThread.java:61)
06-26 17:57:02.892 14254 14254 E EventLogger: playerFailed [0.10]

I also tried this stream, https://drive.google.com/open?id=0BwljeX6541Lud1pzWWx6SWFXRU0 , playback starts, but when I try to enable a subtitle track, I get another exception:

06-26 18:02:11.911 14254 14254 D EventLogger: Tracks [
06-26 18:02:11.911 14254 14254 D EventLogger:   Renderer:0 [
06-26 18:02:11.911 14254 14254 D EventLogger:     Group:0, adaptive_supported=N/A [
06-26 18:02:11.911 14254 14254 D EventLogger:       [X] Track:0, id=1/69, mimeType=video/avc, res=1920x1080, supported=YES
06-26 18:02:11.911 14254 14254 D EventLogger:     ]
06-26 18:02:11.911 14254 14254 D EventLogger:   ]
06-26 18:02:11.911 14254 14254 D EventLogger:   Renderer:2 [
06-26 18:02:11.911 14254 14254 D EventLogger:     Group:0, adaptive_supported=N/A [
06-26 18:02:11.911 14254 14254 D EventLogger:       [X] Track:0, id=1/68, mimeType=audio/mp4a-latm, channels=2, sample_rate=44100, language=eng, supported=YES
06-26 18:02:11.911 14254 14254 D EventLogger:     ]
06-26 18:02:11.911 14254 14254 D EventLogger:   ]
06-26 18:02:11.911 14254 14254 D EventLogger:   Renderer:6 [
06-26 18:02:11.912 14254 14254 D EventLogger:     Group:0, adaptive_supported=N/A [
06-26 18:02:11.912 14254 14254 D EventLogger:       [ ] Track:0, id=1/8261, mimeType=application/cea-608, supported=YES
06-26 18:02:11.912 14254 14254 D EventLogger:     ]
06-26 18:02:11.912 14254 14254 D EventLogger:     Group:1, adaptive_supported=N/A [
06-26 18:02:11.912 14254 14254 D EventLogger:       [ ] Track:0, id=1/70, mimeType=application/dvbsubs, language=eng, supported=YES
06-26 18:02:11.912 14254 14254 D EventLogger:     ]
06-26 18:02:11.912 14254 14254 D EventLogger:   ]
06-26 18:02:11.912 14254 14254 D EventLogger: ]
06-26 18:02:11.912 14254 14254 D EventLogger: videoEnabled [0.20]
06-26 18:02:11.912 14254 14254 D EventLogger: audioEnabled [0.20]
06-26 18:02:11.912 14254 14254 D EventLogger: videoDecoderInitialized [0.20, OMX.Intel.hw_vd.h264]
06-26 18:02:11.912 14254 14254 D EventLogger: videoFormatChanged [0.20, id=1/69, mimeType=video/avc, res=1920x1080]
06-26 18:02:11.913 14254 14254 D EventLogger: audioDecoderInitialized [0.20, OMX.google.aac.decoder]
06-26 18:02:11.913 14254 14254 D EventLogger: audioFormatChanged [0.20, id=1/68, mimeType=audio/mp4a-latm, channels=2, sample_rate=44100, language=eng]
06-26 18:02:11.913 14254 14254 D EventLogger: videoSizeChanged [1920, 1080]
06-26 18:02:11.913 14254 14254 D EventLogger: renderedFirstFrame [Surface(name=null)/@0x10adae8]
06-26 18:02:11.913 14254 14254 D EventLogger: audioSessionId [441]
06-26 18:02:12.012 14254 14254 D EventLogger: state [0.30, true, R]
06-26 18:02:12.473  2083  2083 D CordovaActivity: Stopped the activity.
06-26 18:02:12.522  1434  2050 D omx_vaapi_allocator: i = 0, stride = 1920, offset = 0
06-26 18:02:12.522  1434  2050 D omx_vaapi_allocator: i = 1, stride = 1920, offset = 2088960
06-26 18:02:12.522  1434  2050 D omx_vaapi_allocator: datasize = 3133440
06-26 18:02:12.575  1434  1434 D omx_vaapi_allocator: i = 0, stride = 1920, offset = 0
06-26 18:02:12.575  1434  1434 D omx_vaapi_allocator: i = 1, stride = 1920, offset = 2088960
06-26 18:02:12.575  1434  1434 D omx_vaapi_allocator: datasize = 3133440
06-26 18:02:12.602  1434  1472 D omx_vaapi_allocator: i = 0, stride = 1920, offset = 0
06-26 18:02:12.602  1434  1472 D omx_vaapi_allocator: i = 1, stride = 1920, offset = 2088960
06-26 18:02:12.602  1434  1472 D omx_vaapi_allocator: datasize = 3133440
06-26 18:02:12.632  1434  1434 D omx_vaapi_allocator: i = 0, stride = 1920, offset = 0
06-26 18:02:12.632  1434  1434 D omx_vaapi_allocator: i = 1, stride = 1920, offset = 2088960
06-26 18:02:12.632  1434  1434 D omx_vaapi_allocator: datasize = 3133440
06-26 18:02:13.394 14254 14254 D EventLogger: loading [false]
06-26 18:02:16.594 14254 14254 D EventLogger: loading [true]
06-26 18:02:16.801 14254 14254 D EventLogger: loading [false]
06-26 18:02:17.620 14254 14344 E ExoPlayerImplInternal: Internal runtime error.
06-26 18:02:17.620 14254 14344 E ExoPlayerImplInternal: java.lang.ArrayIndexOutOfBoundsException: length=4; index=6
06-26 18:02:17.620 14254 14344 E ExoPlayerImplInternal: 	at com.google.android.exoplayer2.source.ExtractorMediaPeriod.selectTracks(ExtractorMediaPeriod.java:220)
06-26 18:02:17.620 14254 14344 E ExoPlayerImplInternal: 	at com.google.android.exoplayer2.ExoPlayerImplInternal$MediaPeriodHolder.updatePeriodTrackSelection(ExoPlayerImplInternal.java:1597)
06-26 18:02:17.620 14254 14344 E ExoPlayerImplInternal: 	at com.google.android.exoplayer2.ExoPlayerImplInternal.reselectTracksInternal(ExoPlayerImplInternal.java:848)
06-26 18:02:17.620 14254 14344 E ExoPlayerImplInternal: 	at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:356)
06-26 18:02:17.620 14254 14344 E ExoPlayerImplInternal: 	at android.os.Handler.dispatchMessage(Handler.java:98)
06-26 18:02:17.620 14254 14344 E ExoPlayerImplInternal: 	at android.os.Looper.loop(Looper.java:154)
06-26 18:02:17.620 14254 14344 E ExoPlayerImplInternal: 	at android.os.HandlerThread.run(HandlerThread.java:61)

@ojw28
Copy link
Contributor

ojw28 commented Jun 26, 2017

Thanks for testing. Please sync and try again; I submitted a one line indexing fix that I think resolves this problem.

@ojw28
Copy link
Contributor

ojw28 commented Jun 26, 2017

Fix is here: 076a77e

@goffioul
Copy link
Contributor Author

@ojw28 Thanks, the fixed the exception. Quick question, though. When playing MPEG-TS through HTTP, switching audio/subtitle track is completely smooth, no interruption. However when I do track switching with HLS, I get an interruption. Is there still a seek operation when using HLS?

For the record, my HLS sample stream is that same as the plain MPEG-TS referenced above, but segmented using ffmpeg.

@goffioul
Copy link
Contributor Author

Actually, the commit only targets ExtractorMediaPediod, so obviously it wouldn't affect HLS. So the question should be: will the same changes be applied to HLS too?

@ojw28
Copy link
Contributor

ojw28 commented Jun 26, 2017

Yes, the same benefits will be coming to HLS soon. We're using #2718 for tracking.

@google google locked and limited conversation to collaborators Oct 25, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@ojw28 ojw28

Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@goffioul @ojw28