h264 decoder: bad images

[julianscheel]

We spotted an issue with the h264 decoder where it periodically creates bad images after an initial distortion occured. We have two sample files:
http://jusst.de/files/rpi-issue.ts
http://jusst.de/files/rpi-no-issue.ts

Both files are actually part of the same source files. The difference is that rpi-no-issue.ts is beginning a 100MB later than rpi-issue.ts does.
If you play rpi-no-issue.ts with omxplayer -d rpi-no-issue.ts it play smoothly. If you play rpi-issue.ts with omxplayer -d rpi-issue.ts you see the first distorted image after 28 seconds. After that you periodically see distorted images, which are decoded absolutely fine in the rpi-no-issue.ts.

We can reproduce the same with our VLC tree, so we can be quite sure that it is no demuxing issue.

So for further analysis we compared with playback on host vlc. With rpi-issue.ts there is a discontinuity and decoder error after 28 seconds, which matches the first distortion on the rpi
[h264 @ 0x7f3848de3080] cabac decode of qscale diff failed at 27 45 [h264 @ 0x7f3848de3080] error while decoding MB 27 45, bytestream (41444) [h264 @ 0x7f3848d743a0] reference picture missing during reorder [h264 @ 0x7f3848d743a0] reference picture missing during reorder [h264 @ 0x7f3848d743a0] Missing reference picture, default is 66900 [h264 @ 0x7f3848d743a0] Missing reference picture, default is 66900 [h264 @ 0x7f3848de3080] reference picture missing during reorder [h264 @ 0x7f3848de3080] Missing reference picture, default is 66901 [h264 @ 0x7f3848de3080] mmco: unref short failure [h264 @ 0x7f3848de3080] mmco: unref short failure

With VLC on the host the file continous to plays smoothly afterwards, while on the Raspberry this is the starting point of periodic image distortions with an inverval of 40s.

If you play rpi-no-issue.ts on the rpi it does not show the periodic distortions on the same images which were distorted in the rpi-issues.ts. The difference is just the missing discontinuity at the start of the file.

We see this behavior on several german HD television channels, so it is not specific to this single sample.

[popcornmix]

I can see the issue. I've submitted a bug report to codecs team.

[julianscheel]

Perfect, thanks.

[julianscheel]

Is there any comment from the codec team yet?

[popcornmix]

It has been looked at, but no solution yet.

Something odd is going here.

I'm trying on trickplay on a DK, which can't play .ts files, but when I remux it to mp4 using ffmpeg there's no corruption apart from the 30s part.

However: If I look at the stream in VEGA, it shows a horrible mess at about 1m10s, but doesn't report an error. So it looks like VEGA/us and VLC have some differing interpretation of the standard, in some area which is removed when remuxing. I'll have to investigate more.

[julianscheel]

Just a short update: We tested this with our new MMAL VLC plugins at it behaves exactly the same as it does with omx.

[julianscheel]

Could you maybe also explain what DK and VEGA are? Not that we could help with it, but just for us to understand what you were testing with.

[popcornmix]

DK is development kit - basically a VideoCore IV chip on a test board.

VEGA is commercial software for analysing video streams (http://www.interrasystems.com/analyzers.php). The fact VEGA produces similar corrupt output as we do really suggests the stream is bad, rather than a bug in our code.

However if it's possible to play this stream better (as VLC does), then we'd like to do that (without breaking other compliant streams).

[julianscheel]

Ok. How does VEGA behave with the no-issue.ts? As this is exactly the same bitstream just starting after the initial discontinuity.

[julianscheel]

Any news here?

[julianscheel]

Any updates? Do you need further testing snippets?

[julianscheel]

One more update: We do see this issue with TV channels of the Astra HD+ platform (primary source for german television HD channels) as well. It seems it is some encoder setting which is quite commonly used, which triggers this issue.
@popcornmix You see a chance to get someone to investigate this again in short term?

[julianscheel]

@popcornmix
One more question. You wrote:

but when I remux it to mp4 using ffmpeg there's no corruption apart from the 30s part

What command was used to remux here? I just tried the same, simply doing a

ffmpeg -i rpi-issue.ts -acodec copy -vcodec copy rpi-issue.mp4

The generated file shows the exact same distortions on the Pi which the original ts-file did. Are you sure you only remuxed and not reencoded the stream?
Did you actually try to play your remuxed file on the Pi or only on the DK?

[popcornmix]

Can you try this test build?
https://dl.dropboxusercontent.com/u/3669512/temp/start_dpb.elf

Some info from bug report:

The broken MMCO commands keep the old frame in the DPB until the stream's frame_num values wrap around and come back to the old one, at which point it ends up getting used as a reference.

This is forbidden in the h264 standard, but we aren't detecting invalid streams doing this. CL451051 has a shelved change that seems to fix this on a DK, but I need to make sure it hasn't broken other things.

(later) It passed the hwcodec_test testset

[julianscheel]

We just did some first testings and for all our test-snippets it works fine!
We'll do some more real life testing, but I think this has solved the issue.

Thanks a lot!

[popcornmix]

Should be an official update soon (probably tonight) with this fix in.

[popcornmix]

rpi-update should have the fix. Please confirm it works and close when happy.

[julianscheel]

Looks good. Thanks.