Seamless codec transition midstream

[kgrevehagen]

[REQUIRED] Searched documentation and issues

ExoPlayer Javadoc and code, exoplayer.dev, issue tracker, stack overflow.

[REQUIRED] Question

As mentioned in #7390 I'm trying to initiate a play with a codec without crypto information to avoid an audible glitch because mediaDrm calls are blocking playback.
After fetching the keys I can create a new codec with crypto information set, but I'm having trouble seamlessly switching to that.

In MediaCodecRenderer.java I tried drainAndReinitializeCodec() and the likes at first, which seems to work, but by listening closely(and debugging) it is skipping a lot of buffers in the MediaCodec, around 10 of them it seems.

Then I tried to keep two codecs at the same time, draining the first one before continuing on the 2nd one(manual labor, keeping track of how many buffers were inputted and outputted etc). Depending on the solution I almost got it working, but it still skipped 1 or 2 MediaCodec buffers.

In my mind, this should theoretically work, but I just can't get it to work. Do you know if this is theoretically possible? I only had two codecs, but they were using the same of everything else, like buffer, outputBufferInfo, inputIndex, outputIndex etc).
The 2nd codec is created and started while the 1st is still used to feed, so there is no issue in that aspect, in regards of setup/creation/configration time etc. But, it seems that when I do the switch to the 2nd codec, the last two buffers that I queue is not found when dequeueOutputBuffer is called. It simply returns -1.

I can provide code samples for the latter approach if you need it to answer the question.

To sum up: I want to seamlessly switch from one codec to another while playing and without loosing any information so it is without any glitches at all(basically the same codec, just two different instances and the 2nd one has MediaCrypto sent to it, instead of null).

I'm happy to implement it myself, I just need to know if this is theoretically possible, or even possible with the existing code, or some guidance on how it can be achieved.

I really hope this is doable, it would make DRM playback awesome with ExoPlayer :)

If I can get some guidance, I'm happy to contribute with the fix.

Thanks a lot in advance!

A full bug report captured from the device

N/A

Link to test content

Any DASH with Widevine(and most likely any drm-ed content with clear lead)

[tonihei]

The issue with the audible glitch is caused by the blocking mediaDrm calls in the renderer as you correctly identified in #7390. We have a pending improvement plan to move the entire DRM session acquisition to the Loader (that runs on its own thread), so that this does not occur anymore. Note that it not only affects clear lead playbacks, but also playlists where we change drm protections or switch from clear to protected playback. This effort is tracked by #4133.

If we implemented the mentioned improvement of moving the DRM session calls to another thread, then I guess your problem would be solved without using a second MediaCodec. However, we currently paused further development in this area because it turned out to be quite complicated :).

Coming back to your original question whether the proposed two MediaCodec solution could work, I think the short answer is yes. You should be able to feed all clear samples to one decoder (followed by an end-of-stream signal) and start feeding all encrpyted samples to another decoder without losing any of them. The tricky bit is to attach the output surface to the right decoder at the right time and to manage the release timing of these samples. We probably can't provide deep guidance on how to do this as we would need to investigate ourselves first. But in theory, you should be able to get this to work.

But, it seems that when I do the switch to the 2nd codec, the last two buffers that I queue is not found when dequeueOutputBuffer is called. It simply returns -1.

That may as well depend on the media. For example if the codec waits for further input to be able to decode these samples or because you haven't queued the end of input signal.

[tonihei]

Closing because the question was answered above. Feel free to reopen if not.