CEA-608 stream does not conform to H.264 standards

[ArktekniK]

Hello,

https://livesim.dashif.org/dash/vod/testpic_2s/cea608.mpd

This stream is non-conformant to H.264. The stream is also supplied in the DASH.js test player list of available streams. DASH.js seems to be able to play this non-conformant stream without issue.

The above stream contains two channels of CEA-608 closed caption data which is correctly contained within NAL units of type 0x06 - Supplemental Enhancement Information. However, the order of the NALs does not conform to the H.264 specification.
Specifically, the passage 7.4.1.2.3 Order of NAL units and coded pictures and association to access units

The following constraints shall be obeyed by the order of the coded pictures and non-VCL NAL units within an access unit:
– When an access unit delimiter NAL unit is present, it shall be the first NAL unit. There shall be at most one access unit delimiter NAL unit in any access unit.
– When any SEI NAL units are present, they shall precede the primary coded picture.
...

(Emphasis mine)

Please see the first segment analysis screenshot below (Produced from an internal tool).
SEI NAL units number 4 and 5 should come before the primary coded picture, which is NAL number 3.

As this is the only content available that has multiple CEA-608 channels/languages embedded within the video sream, it would be great to have it resolved.

Thanks!

[ArktekniK]

Forgot to mention that additionally, the live version of this stream is also affected in the same way:
https://livesim.dashif.org/livesim/testpic_2s/cea608.mpd

[TobbeEdgeware]

Yeah, that is correct. That's a risk of making test assets by hand and also implementing the player ;-)
Nobody has noticed in over 4 years!

It's an old sin in (https://github.com/Dash-Industry-Forum/dash-live-source-simulator/blob/develop/dashlivesim/cc_inserter/cc_inserter.py) which inserts SEI nal units in the video samples.
It inserts them at the end, but putting them at the start after any SPS/PPS would be the right thing to do.

I'll look into updating both the code and the asset. The live stream is the same content, just shifted in time and sequence number, so if the VoD asset is fixed, the live one will be OK at the same time.

[ArktekniK]

That's great, thanks for confirming that.
I did take a look at cc_inserter.py but unfortunately didn't have the time to investigate further and put together a PR.
FYI In my particular case our native player didn't pick up the CCs because of an early-out as soon as the primary encoded picture is found.

[TobbeEdgeware]

Should be fixed now. I wrote a small program to move the NAL units to the right position in the samples since I didn't have the original template for generating scc files around. I haven't fixed the cc_inserter.py, so I made #99 for that.

[micolous]

BTW, it also looks like the SEI is truncated. ffprobe reports:

[h264 @ 0x556d308e86c0] SEI type 4 size 424 truncated at 423

Comcast's Caption Inspector tool (which uses ffmpeg to read SEIs) also reports:

ERROR DBG_FF_MPEG [FFMPEG:0] - SEI type 4 size 424 truncated at 423

In the case of an A/53 CC SEI, the last byte is a padding byte, but it is correct for a player to ignore these broken SEIs.

Similarly, VLC 3.0.16 does not report captions in the video files. I'm able to play other H.264 video files with 608 or 708 captions fine.

[TobbeEdgeware]

Hmm. I also see the message when I run ffprobe, but it is a bit strange since the SEI NAL units of type 4 are 56 bytes long, thus not even close to 424 bytes long.

However, there are actually 3 SEI NAL units at the beginning of a segment. First a 754B NAL unit from VLC and then two 56-bytes NAL units with CC1 and CC3 CEA-608 data. In total that is 866B = 2 x 433, so it seems that ffprobe/ffmpeg somehow messes up the different SEI NAL units.

Anyway, the CEA-608 NAL units are not proper since they lack the rbsp_trailing_bits (a last 0x80 byte for this byte-aligned payload), so that should be added to them. I'll look into adding that.

[TobbeEdgeware]

There were actually two issues with the CEA-608 SEI NAL units in the asset.

1. They end with an 0xff byte instead of an 0x80 rbsp_trailing_bits byte as it should
2. The payload length included that byte which it shouldn't

I've fixed these two bytes in each CEA-608 NAL unit in the segments, so now the content should be fine.
I've also added some SEI NAL unit parsing to the mp4ff open source project, to verify that the NAL units are now fine:
Eyevinn/mp4ff#116

[enen92]

@TobbeEdgeware @tobbee were the public samples actually fixed? Running mp4ff against an mp4 download of https://livesim.dashif.org/dash/vod/testpic_2s/cea608.mpd still reports the SEI NALUs after the primary coded picture.

while a conformant file reports them before:

Recent ffmpeg versions complain about the "Late SEI" and don't even consider the elements as part of the picture.

Regarding the 0xFF removal of the last comment it was likely the marker_bits...
I've opened #115 for that particular issue