CMAF DASH, availabilityTimeOffset and CMAF HLS support

[bencecsire]

Dear Support,

i would like to ask your help

System info

Ubuntu 18.04
Packager version 7973c53-release

Issue and steps to reproduce the problem

I would like to create CMAF DASH and HLS stream, but I could not yet. I would like to use this workflow: FFmpeg as an encoder, shaka packager, nginx as server and bitmovin as player. I have already configured bitmovin, it is working for sure. Also I believe ffmpeg encoder is working for DASH too, but here is the command I use:

ffmpeg
-f decklink
-i 'DeckLink Duo (1)'
-pix_fmt yuvj420p
-profile:v Main
-preset ultrafast
-tune zerolatency
-g 50
-keyint_min 25
-b:v 4000k
-s 1920x1080
-c:a libfdk_aac
-c:v libx264
-method PUT
-streaming 1
-http_persistent 1
-f mpegts udp://127.0.0.1:40000

I believe packager or nginx could be the problem.

Packager Command:
/home/mediaadmin/shaka_packager/src/out/Release/packager
'in=udp://127.0.0.1:40000,stream=0,init_segment=a_init.m4s,segment_template=a$Number$.m4s'
'in=udp://127.0.0.1:40000,stream=1,init_segment=b_init.m4s,segment_template=b$Number$.m4s'
--segment_duration 6
--mpd_output manifest.mpd

Right now I just tried with this simple packager command. The created manifest is not providing the "availabilityTimeOffset" value, which is I believe needed. Can you tell me what am I missing?

Also I have another question. Am I able to create low latency cmaf hls with the setup I mentioned, with Shaka packager?

Br,
Bence

[kqyang]

@bencecsire availabilityTimeOffset is not needed. It is only useful if an offset is needed to adjust the segment availability time. While in Shaka Packager, we already set MPD@availabilityStartTime appropriately so an additional offset is not needed.

You may check out our sister project: https://github.com/google/shaka-streamer, which wraps the complexities of FFmpeg and Shaka Packager and prepares streaming media for you using config-file based approach.

Am I able to create low latency cmaf hls with the setup I mentioned, with Shaka packager?

No, we do not support low latency cmaf HLS yet. What is the latency you are looking for?

[bencecsire]

Thanks for the information, I am going to check shaka-streamer.

I'm looking for about 5 sec latency. I would like to test cmaf to reduce encoder and storage cost also, not only for reduce latency, so I don't need to achive too low.

[kqyang]

Ok. Shaka Packager supports CMAF, but it does not support Low Latency HLS yet.

[jbree]

@kqyang My understanding is that, to support low-latency DASH using CMAF + chunked encoding, shaka-packager would only need to add support for the BaseURL@availabilityTimeOffset and the BaseURL@availabilityTimeComplete parameters.

These parameters conspire to inform the client that fragments will be ready prior to the completed segment. I suppose the truth of this declaration depends on whether or not shaka-packager writes individual fragments to disk prior to segment completion (and obviously also upon the http server).

I may be interested in trying to add support for this. Are my assumptions about this correct? Do you have any other insights and guidance on implementation?

[aleek]

As far as I remember, shaka creates fragments with different name, renaming them after completion. So along with BaseURL@availabilityTimeOffset, you'll have to change this behaviour. Also, I think that http server should be configured to support Partial Content.

[jbree]

@aleek Thanks, I'll look into this also.

[kqyang]

@jbree @aleek Thanks for bring this up. Yes, @aleek is correct that the writing behavior needs to be changed too.

In summary, here are the items I think that needs to be implemented to get a complete low-latency DASH support (may be incomplete):

1. Support BaseURL@availabilityTimeOffset and BaseURL@availabilityTimeComplete attributes as @jbree mentioned

2. Change the writing behavior to create the new segment once the first fragment (a.k.a chunk in CMAF) is ready and update the MuxerListener. Here is the existing logic: https://github.com/google/shaka-packager/blob/master/packager/media/formats/mp4/multi_segment_segmenter.cc.

3. We also need to support uploading directly to http for serving, i.e. we need HTTP PUT output #149.

And @jbree, you are welcomed and appreciated to work on 1 and 2. Let us know if you have any questions.

[jbree]

@kqyang Thanks for the outline. I've made some modifications to MultiSegmentSegmenter to make it write subsegments as they arrive at base Segmenter.

--generate_sidx_in_media_segments needs to be disabled in this mode since we don't have enough information when the first segment is written.

I wonder if you have any thoughts on properly signalling the MuxerListener. Currently I am writing out each of the subsegments, calling muxer_lister()->OnKeyFrame() as appropriate, and then only calling muxer_listener()->OnNewSegment() when the whole segment is finished. This, of course, means that the MPD isn't updated until the segment is completed. I'm not sure if there's an existing mechanism for predicting the segment duration in the MPD, or a way to update it after the fact if the prediction was wrong.

Thanks in advance.

[jbree]

After some more research, I think the way to get around the problem in my previous post is to resolve #554. Then the client can anticipate the segments without a listing in the MPD.

[kqyang]

@jbree See my replies below.

--generate_sidx_in_media_segments needs to be disabled in this mode since we don't have enough information when the first segment is written.

Yes, --generate_sidx_in_media_segments has to be disabled.

then only calling muxer_listener()->OnNewSegment() when the whole segment is finished

I think we should split it into two: OnNewSegmentStart() and OnNewSegmentComplete(). OnNewSegmentStart() should be called when the first chunk is ready and OnNewSegmentComplete() should be called after all chunks are ready.

An expected segment duration should be provided in OnNewSegmentStart(), which should be the same as the actual segment duration in practice if the stream has constant GOP duration.

After some more research, I think the way to get around the problem in my previous post is to resolve #554.

I don't think it is a requirement for this issue. I am interested in your reasoning behind it.

[jbree]

Perhaps not a requirement, but I think the path of least resistance might be to create a static live MPD. The ULL examples I have seen use this method. Given a static live MPD with properly created fixed duration segments, we don't need to worry about updating the MPD at segment start with the expected segment duration.

Even with a constant GOP duration, I have found that my first segment is always shorter than the successive segments because the PTS doesn't always start at 0.

Implementing both static and dynamic mechanisms is possible, of course.

Supposing we add OnNewSegmentStart() and OnNewSegmentComplete(), we would need to update the MPD at segment start with the new segment and expected duration. Do you anticipate any client issues in the case when a segment is updated with a different duration?

[kqyang]

I see. I agree with your assessment. There are less uncertainty regarding player handling on the latest segment duration.

You are welcomed to tackle #554 first which I don't think it it difficult to address. Let me know if you need pointers. OTOH, I do like to see low latency to be supported regardless of whether SegmentTemplate duration is used.

Do you anticipate any client issues in the case when a segment is updated with a different duration?

Frankly I don't know. Do you know of any player that support low latency DASH already? I wonder if they allow the latest segment duration to be updated, which seems logical to me and may not be difficult to implement in player side.

[probablybenallen]

Do you know of any player that support low latency DASH already?

The DASH forum player now supports Low Latency Mode in the options:
https://reference.dashif.org/dash.js/v3.1.3/samples/dash-if-reference-player/index.html

No idea if there are any of the issues you describe with it though I'm afraid

[SupritiHyland]

I am curious about live packaging for low latency HLS. When can we expect this feature support for Shaka Packager?

[truong-hua]

I am curious about live packaging for low latency HLS. When can we expect this feature support for Shaka Packager?

I'm looking for LL-HLS with CMAF too