RTMP video only file playback issue with ExoPlayer2

[Srinu-try]

Issue description

Hi All,

I am facing a playback issue in playing Video only RTMP stream using ExoPlayer2 RTMP extension.

I am streaming a video only file with FFMPEG using Wowza server (see the below command),

ffmpeg -i input.mp4 -c copy rtmp://WowzaServerIPADDRESS:1935/live/Cam

Exoplayer2 is failing to parse the AVC decoder SPS/PPS data and failed at "AvcConfig avcConfig = AvcConfig.parse(videoSequence);" API. I found that SPS/PPS data from the RTMPClient is missing last 13 bytes of information.

"rtmpClient.read(buffer, offset, readLength)" API output contains 13 bytes of less data (missing 9 bytes of SPS/PPS data + 4 bytes of Skip header data before the next Video tag header)

When I use a lower bitrate (<5 Mbps 3840*1920 video resolution) RTMP streaming, I observed similar behaviour. Here 13 bytes of data is mising from the 1st video frame received from RTMPClient instead of the data from SPS/PPS. In this case, 1st frame is displayed properly, but as 13 bytes are missed from 1st frame, the offset of next Video tag header is wrong and ExoPlayer fails to understand the next Video tag header offset position.

Surpraisingly, this issue is happening only with Video only file streaming. If Audio is present, this 13 bytes missing scenario is not happening.

Request your help in this regard.

Reproduction steps

Use the below command with video only input file for RTMP streaming and play using ExoPlayer2 with RTMP support

ffmpeg -re -i input.mp4 -c copy -f flv rtmp://WowzaServerIPADDRESS:port/live/Cam

Link to test content

Video File:
Download "bbb_sunflower_2160p_30fps_normal.mp4" file from
https://download.blender.org/demo/movies/BBB/

and extract video using FFMPEG

ffmpeg -i bbb_sunflower_2160p_30fps_normal.mp4 -c copy -an bbb-VideoOnly.mp4

Another video only file is uploaded at
https://www.dropbox.com/s/q7gyqk02ac7ag0x/Cam2_7Mbps.mp4?dl=0

Version of ExoPlayer being used

Using the latest code from release-v2 branch of ExoPlayer2 project.

Device(s) and version(s) of Android being used

I am using Samsung Note 8 device with Android version 7.1.1

A full bug report captured from the device

E/LoadTask: Unexpected exception loading stream
            java.lang.IllegalArgumentException
                at com.google.android.exoplayer2.util.Assertions.checkArgument(Assertions.java:37)
                at com.google.android.exoplayer2.util.ParsableByteArray.setPosition(ParsableByteArray.java:146)
                at com.google.android.exoplayer2.util.ParsableByteArray.skipBytes(ParsableByteArray.java:158)
                at com.google.android.exoplayer2.video.AvcConfig.buildNalUnitForChild(AvcConfig.java:93)
                at com.google.android.exoplayer2.video.AvcConfig.parse(AvcConfig.java:60)
                at com.google.android.exoplayer2.extractor.flv.VideoTagPayloadReader.parsePayload(VideoTagPayloadReader.java:90)
                at com.google.android.exoplayer2.extractor.flv.TagPayloadReader.consume(TagPayloadReader.java:65)
                at com.google.android.exoplayer2.extractor.flv.FlvExtractor.readTagData(FlvExtractor.java:276)
                at com.google.android.exoplayer2.extractor.flv.FlvExtractor.read(FlvExtractor.java:169)
                at com.google.android.exoplayer2.source.ExtractorMediaPeriod$ExtractingLoadable.load(ExtractorMediaPeriod.java:853)
                at com.google.android.exoplayer2.upstream.Loader$LoadTask.run(Loader.java:308)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)

[ojw28]

The reproduction steps provided appear to be quite time consuming. Please can you provide more direct reproduction steps, such as an rtmp test stream that we can use to test this without having to spend time using FFmpeg / Wowza streaming engine ourselves? We really don't have sufficient time to do this kind of thing to debug non-widespread issues.

[Srinu-try]

Hi,

Thanks for your response.

Finally, I got a public IP to stream the RTMP video.
You can stream the content and reproduce the issue using the below address,
rtmp://76.81.29.11:1935/live/Cam2

The stream restarts every 10 mins. During the restart, RTMP server is taking some time to connect with the RTMP clients.

Thanks,
Sri

[ojw28]

The stream appears malformed to me, although I'm not sure whether there's something I'm missing. What I see when I attach a debugger is:

• First FLV tag has type SCRIPTDATAOBJECT, and is well formed (fine)
• Second FLV has type VIDEODATA (fine)
• In the VIDEODATA packet: CodecId = 7 (AVC), AVCPacketType = 0 (Avc Sequence Header), CompositionTime = 0 (all fine)
• The remainder of the packet should then be a AVCDecoderConfigurationRecord as defined in ISO 14496-15. This appears to be malformed, in particular from numOfPictureParameterSets, which has value of 174 (not fine).

[ojw28]

• If I dump the stream to a file using rtmpdump and then try and play back that file, I do not see invalid data and the stream is able to play fine.
• If I dump the stream using LibRtmp and then try and play back that file, I do see invalid data and the failure occurs.

So it appears the stream is not malformed, but there's probably a bug in LibRtmp that means it's not being read correctly. I'll file an issue on LibRtmp.

[Srinu-try]

Hi,

Thanks for raising the issue with libRtmp.

Yes. LiRtmp library is causing the issue here. The output buffer data from libRtmp is missing last 13 bytes of SPS/PPS header data. Length of the data is shown as 43 bytes but libRtmp is providing only 30 bytes of data from the respective header. The rest of 13 bytes are filed from next Video Tag data header which is causing the issue.

[ojw28]

The underlying issue is tracked by ant-media/LibRtmp-Client-for-Android#70