Live HLS stream does not play on flashls v0.3.3 and later

[hohno-panopto]

Live stream which my program generates does not play flashls v0.3.3 or the latest version.
It plays fine on v0.3.2 or before.
Example: http://panopto-public-scratch.s3.amazonaws.com/live_sample_1/master.m3u8
This is not real live, but it does not have end tag to simulate live HLS stream.
I tested with various version of choromeless flash swf.

[mangui]

Hi @Hiroshi-p,
indeed I can repro, it should most probably be related to the changes introduced by 1cbd963
related to #101
will check soon
Cheers, Mangui

[mangui]

Hi @Hiroshi-p would you mind rechecking with latest swf on dev branch ?

[hohno-panopto]

Hi @mangui
I tried both release build and debug build of flashlsChromeless.swf of commit e38f04b

It played audio part only once or twice, but most of the time it still did not play the sample stream.
(http://panopto-public-scratch.s3.amazonaws.com/live_sample_1/master.m3u8)

Note that I tried v0.3.2. again and confirmed that this sample stream still works.

[mangui]

@Hiroshi-p it looks like there is no AUD NAL unit in your stream, it is not compliant.
this is messing up flashls.
see below :

in theory we should have
"DEBUG:AVC: AUD, SPS, PPS, SEI, SEI, IDR slices" chromeless
..
"DEBUG:AVC: AUD, SEI, NDR slices" chromeless

and we have
"DEBUG:AVC: SPS, PPS, SEI, SEI, IDR slices" chromeless
"DEBUG:AVC: width/height:1280/720" chromeless
"DEBUG:AVC: H264 Main level 31" chromeless
"DEBUG:AVC: SEI, NDR slices" chromeless
"DEBUG:AVC: SEI, NDR slices" chromeless
"DEBUG:AVC: SEI, NDR slices" chromeless
"DEBUG:AVC: SEI, NDR slices" chromeless
"DEBUG:AVC: SEI, NDR slices" chromeless
"DEBUG:AVC: SEI, NDR slices"

[mangui]

plz recheck on your hand, thanks

[hohno-panopto]

@mangui,
Thank you for the analysis.
I looked at our stream.
I see AUD exists at the end of each video sample, rather than start of it.
That's how the encoder which we use generate.
I suppose your change at v0.32->v0.33 started to ignore the trailing AUD?

[mangui]

mm interesting, in fact any AUD located after a IDR or NDR is ignored and considered as being part of the IDR/NDR (for NAL unit parsing optimization purpose).
conditional break here: https://github.com/mangui/flashls/blob/dev/src/org/mangui/hls/demux/Nalu.as#L61
i changed some parsing logic between 032 and 033 that introduce this bug.
I will see what i can do but most probably this should be the root cause of this issue...

[mangui]

@Hiroshi-p issue should be fixed now.
in case IDR/NDR unit is found but AUD has not been found, I don't break NAL parsing and I continue looking for new NALu(s).
this will fix your stream, but please note that inserting AUD at the end of the PES is not good performance-wise, as NALu parsing is time consuming.

[hohno-panopto]

@mangui ,
Thank you for the fix.
I confirmed that our stream now works by flashlsChrome.swf of commit e0b89a2
As you pointed out, this seems to cause some CPU usage associated.
Here is the comparison by using flashlsChromeless as of today vs. v0.3.2.

Looks like some CPU increment from v0.3.2 to the current exists also with your sample stream (http://www.streambox.fr/playlists/test_001/stream.m3u8), although it is less severe than my sample.

I wonder if this is actually some general overhead introduced by the parser change between v0.3.2 and now?

I want to learn your thought if there is a chance that we will have general improvement in flahsls side sometime -or- our stream is very exotic format that is never seen elsewhere and we should definitely fix the source.

[mangui]

extracting NAL units from a PES packet is a tedious process.
each NAL unit is delimited by its start sequence 00 00 01 or 00 00 00 01.
in order to avoid parsing the whole PES packet, the initial assumption was that IDR/NDR (the biggest NAL units) were the latest NAL unit in a PES packet.
I changed that, so that in case no AUD has been found, it will continue looking for other NAL units after IDR/NDR.
I don't think the overhead difference you see between v0.3.2 and dev is related to this.
chromeless page contains a lot more ExternalInterface call (Flash to JS) and this is generally eating a bunch of CPU. also the chromeless web page contains a bunch of div that are refreshed upon flash event reception.
if you want to compare CPU wise v0.3.2 against dev I would recommend to use FlowPlayer or GrindPlayer.

[mangui]

at least you should observe similar performance with the original test stream. but not with yours

[hohno-panopto]

@mangui ,
I tested v0.3.2 vs. current with GrindPlayer test page and observed very similar CPU usage.
I agree that the difference with chromeless is something unrelated to this NAL parsing.
Thanks again.
flashls is really a great software!