Mix couple videos

[alxgord]

Can I receive two frames from two video and mix it in custom renderer?

[andrewlewis]

If you don't need to video frame output to be perfectly synchronized, you could try having two players output to TextureViews then set the alpha on the top view, but I'm not sure if this gives the type of mixing you're after.

A more flexible approach would be to have two players, where each one is configured to output to its own SurfaceTexture. These off-screen textures can be used as external texture inputs in a GL shader that blends them and outputs to a single surface view. Your code is responsible for synchronization of the two video streams in this setup. Giving detailed instructions is outside the scope of this tracker but there is some info about getting the frame presentation timestamp of the image in the surface texture in #5860. Also, if you need your implementation to be compatible across lots of Android devices, limitations on using multiple video decoders concurrently may be relevant, so see #1286.

[ArshakManukyan]

@andrewlewis I tried this demo application, and faced with an error while trying to use it in my app. I'll provide stacktrace below, and seems renderer calls reset() -> .... -> mediacodec.stop()
and there throws IllegalStateException. Is this is a bug or am doing something wrong ?

please see above for more details ))

 private static int genSurfaceTexture() {
    int[] args = new int[1];
    GLES20.glGenTextures(args.length, args, 0);
    int textureHandle = args[0];
    GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, textureHandle);
    GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
    GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
    GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
    
    return textureHandle;
  }

and usage

      SurfaceTexture surfaceTexture = new SurfaceTexture(genSurfaceTexture());
      surfaceTexture.setOnFrameAvailableListener(new SurfaceTexture.OnFrameAvailableListener() {
        @Override
        public void onFrameAvailable(SurfaceTexture surfaceTexture) {

          System.out.println();
        }
      });
      player.setVideoSurface(new Surface(surfaceTexture));

there is a exception

2019-12-01 20:12:51.342 16268-16661/com.google.android.exoplayer2.demo E/ExoPlayerImplInternal: Reset failed.
    java.lang.IllegalStateException
        at android.media.MediaCodec.native_stop(Native Method)
        at android.media.MediaCodec.stop(MediaCodec.java:2084)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.releaseCodec(MediaCodecRenderer.java:613)
        at com.google.android.exoplayer2.video.MediaCodecVideoRenderer.releaseCodec(MediaCodecVideoRenderer.java:596)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.onReset(MediaCodecRenderer.java:591)
        at com.google.android.exoplayer2.video.MediaCodecVideoRenderer.onReset(MediaCodecVideoRenderer.java:460)
        at com.google.android.exoplayer2.BaseRenderer.reset(BaseRenderer.java:164)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.resetInternal(ExoPlayerImplInternal.java:860)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.stopInternal(ExoPlayerImplInternal.java:813)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:399)
        at android.os.Handler.dispatchMessage(Handler.java:102)
        at android.os.Looper.loop(Looper.java:214)
        at android.os.HandlerThread.run(HandlerThread.java:65)

[shahen94]

@ArshakManukyan having the same issue