Omit SegmentTimeline completely, with allow_approximate_segment_timeline option

[jatinderkumargupta]

System info

Operating System: Ubuntu 16.04
Shaka Packager Version: v2.1.0

Issue and steps to reproduce the problem

Packager Command:
packager
'input=udp://192.168.2.117:1234?reuse=1,stream=audio,init_segment=/path/audio_init.mp4,segment_template=/path/audio_$Number$.m4s'
'input=udp://192.168.2.117:1234?reuse=1,stream=video,init_segment=/path/video_init.mp4,segment_template=/path/video_$Number$.m4s'
--segment_duration 1
--minimum_update_period 15
--min_buffer_time 1
--suggested_presentation_delay 1
--time_shift_buffer_depth 1
--preserved_segments_outside_live_window 10
--allow_approximate_segment_timeline
--segment_template_constant_duration
--mpd_output /path/FromUDP.mpd

What is the expected result?
<SegmentTemplate timescale="90000" initialization="video_init.mp4" media="video_$Number$.m4s" startNumber="11" duration="48000"/>

What happens instead?
<SegmentTemplate timescale="90000" initialization="video_init.mp4" media="video_$Number$.m4s" startNumber="11"> <SegmentTimeline> <S t="5893656499" d="91918"/> <S t="5893748417" d="90091"/> </SegmentTimeline> </SegmentTemplate>

When we generate DASH package from UDP, with option:
--allow_approximate_segment_timeline --segment_template_constant_duration

It should generate MPD with: <SegmentTemplate ... duration="48000" />
But it generates <SegmentTimeline> in the MPD output.

[kqyang]

@jatinderkumargupta Thanks for reporting the problem. Can you try the latest release v2.3.0?

[jatinderkumargupta]

@kqyang I tried with version v2.3.0-5bf8ad5-release. I am getting same problem.

[kqyang]

Thanks. We'll look into it.

[kqyang]

@jatinderkumargupta What is the GOP size of your content? Does your content have constant GOP size?

constant duration is generated only if the segment duration specified aligns with GOP.

[jatinderkumargupta]

@kqyang Please check the generated MPD template below, will this help to get GOP size?

<?xml version="1.0" encoding="UTF-8"?>
<!--Generated with https://github.com/google/shaka-packager version v2.1.0-cf40acc-release-->
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xsi:schemaLocation="urn:mpeg:dash:schema:mpd:2011 DASH-MPD.xsd" xmlns:cenc="urn:mpeg:cenc:2013" profiles="urn:mpeg:dash:profile:isoff-live:2011" minBufferTime="PT3S" type="dynamic" publishTime="2019-02-12T07:33:47Z" availabilityStartTime="2019-02-11T16:36:12Z" minimumUpdatePeriod="PT1S" timeShiftBufferDepth="PT10S" suggestedPresentationDelay="PT1S">
  <Period id="0" start="PT0S">
	<AdaptationSet id="0" contentType="audio" segmentAlignment="true">
	  <Representation id="0" bandwidth="132998" codecs="mp4a.40.2" mimeType="audio/mp4" audioSamplingRate="48000">
		<AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
		<SegmentTemplate timescale="90000" initialization="audio_init.mp4" media="audio_$Number$.m4s" startNumber="104">
		  <SegmentTimeline>
			<S t="4845780566" d="90000"/>
			<S t="4845870566" d="85920"/>
			<S t="4845960326" d="90000" r="9"/>
		  </SegmentTimeline>
		</SegmentTemplate>
	  </Representation>
	</AdaptationSet>
	<AdaptationSet id="1" contentType="video" width="720" height="480" frameRate="90000/1501" segmentAlignment="true" par="16:9">
	  <Representation id="1" bandwidth="1458437" codecs="avc1.4d401f" mimeType="video/mp4" sar="32:27">
		<SegmentTemplate timescale="90000" initialization="video_init.mp4" media="video_$Number$.m4s" startNumber="105">
		  <SegmentTimeline>
			<S t="4845885924" d="90091" r="10"/>
		  </SegmentTimeline>
		</SegmentTemplate>
	  </Representation>
	</AdaptationSet>
  </Period>
  <UTCTiming schemeIdUri="urn:mpeg:dash:utc:ntp:2014" value="ntp.ubuntu.com"/>
</MPD>

[kqyang]

@jatinderkumargupta Yes, they have the same GOPs.

I know where the problem is now. To use constant duration in SegmentTemplate, the segment start time needs to be approximately: SegmentNumber * duration. However, right now the segment number always starts from 1 when the packaging starts. We need to update the logic so the initial segment number equals to SegmentStartTime / duration.

https://github.com/google/shaka-packager/blob/8d11e5ea6494dd70513b38d7c1d5bb1b42a56e8c/packager/mpd/base/xml/xml_node.cc#L53 This code needs to be updated to check for approximately equal instead.

[jatinderkumargupta]

@kqyang how much variation it should consider equal?

You mean the difference in timescale="90000" and d="90091" in SegmentTimeline.

[kqyang]

Right now the threshold is 5%. We can add a flag to allow it to be customized.

[jatinderkumargupta]

@kqyang
I added following logs before
Line 53 in 8d11e5e in file shaka-packager/packager/mpd/base/xml/xml_node.cc

LOG(INFO) << "First Segment Start Time: " << first_segment.start_time;
LOG(INFO) << "First Segment Duration: " << first_segment.duration;
LOG(INFO) << "Start Number: " << start_number;

Following is the output:
First Segment Start Time: 2584554174
First Segment Duration: 67200
Start Number: 1
First Segment Start Time: 2584554174
First Segment Duration: 67200
Start Number: 1
First Segment Start Time: 2584630704
First Segment Duration: 90090
Start Number: 1
First Segment Start Time: 2584554174
First Segment Duration: 67200
Start Number: 1
First Segment Start Time: 2584630704
First Segment Duration: 90090
Start Number: 1
First Segment Start Time: 2584621374
First Segment Duration: 90240
Start Number: 2
First Segment Start Time: 2584630704
First Segment Duration: 90090
Start Number: 1
First Segment Start Time: 2584621374
First Segment Duration: 90240
Start Number: 2
First Segment Start Time: 2584630704
First Segment Duration: 90090
Start Number: 1
First Segment Start Time: 2584621374
First Segment Duration: 90240
Start Number: 2
First Segment Start Time: 2584720794
First Segment Duration: 90090
Start Number: 2

I think the packager expects the start time to be zero, when it starts. Based on these logs, it will always return false, and will always add SegmentTimeline in the MPD file.

[kqyang]

Yes, see the comments in #554 (comment).

[jatinderkumargupta]

@kqyang is anybody working on this fix?

[kqyang]

@jatinderkumargupta We don't have resource to work on it in this milestone (v2.4). We'll look into it in v2.5.

[sr1990]

@kqyang I was able reproduce this issue and following are the changes that I tried to resolve the issue: 

1. In Representation::AddSegmentInfo:  https://github.com/google/shaka-packager/blob/069dbc82f7d65b98064670066abe83e264e747f7/packager/mpd/base/representation.cc#L334 added:

if (segment_infos_.empty()) {                                                                                                           
  start_number_ = start_time / duration;                                  
}  

2. In IsTimelineConstantDuration(): 
   Added Approximate check with 5% threshold at: https://github.com/google/shaka-packager/blob/8d11e5ea6494dd70513b38d7c1d5bb1b42a56e8c/packager/mpd/base/xml/xml_node.cc#L53
   Also changed the line to:
   ApproximiatelyEqual(first_segment.start_time, first_segment.duration * start_number)

Stream that I am testing has constant GOP with segment duration = 10s and time_shift_buffer_depth set to 30 seconds set with options: --mpd_output live.mpd --allow_approximate_segment_timeline --segment_template_constant_duration --segment_duration 10 --time_shift_buffer_depth 30

And now IsTimelineConstantDuration() returns false after: https://github.com/google/shaka-packager/blob/8d11e5ea6494dd70513b38d7c1d5bb1b42a56e8c/packager/mpd/base/xml/xml_node.cc#L60

as the last_segment is repeated 4 times in SegmentTimeline in the generated mpd mentioned below.
Was able to generate the expected mpd when I tried (!(last_segment.repeat == 0 || first_segment.repeat == 0)) instead.

MPD

        <SegmentTemplate timescale="90000" initialization="live-video.mp4" media="live-video-$Number$.mp4" startNumber="28534">
          <SegmentTimeline>
            <S t="5136186497" d="180000"/>
            <S t="5136366497" d="900000" r="4"/>
          </SegmentTimeline>

Also as per DASH IOP, "Segments should have almost equal duration. The maximum tolerance of segment duration shall be ±50% and the maximum accumulated deviation over multiple segments shall be ±50% of the signaled segment duration (i.e. the @duration). " Do you want to add the flag that you have mentioned above for this reason?

Please let me know what you think about the above changes.
   

[sridhard]

@sr1990 @kqyang this issue is same as the issue I am facing #744 . @sr1990 were you able to generate mpd without segment timeline?

[kqyang]

@sr1990 Thanks for looking into this issue. It seems like there are problems in your source content:

            <S t="5136186497" d="180000"/>
            <S t="5136366497" d="900000" r="4"/>

Are you always seeing the first segment to have 20 seconds instead of 10 seconds?

[sr1990]

@sr1990 Thanks for looking into this issue. It seems like there are problems in your source content:

            <S t="5136186497" d="180000"/>
            <S t="5136366497" d="900000" r="4"/>

Are you always seeing the first segment to have 20 seconds instead of 10 seconds?

@kqyang I sometimes see

 <S t="985758711" d="720000"/>
 <S t="986478711" d="900000" r="2"/>

and also

 <S t="978918711" d="360000"/>
 <S t="979278711" d="900000" r="2"/>

The content that I am using has key IDR frame every 48 frames and I am using VLC to create a UDP stream. Should I use some other tool to stream? Also please let me know how I can share the content.

[sridhard]

@kqyang @sr1990 the first should always be of proper duration? for example if we have the segment duration of 1sec and timescale is 90000, then the first audio segment should always be 90000?

In the issue #744 I am facing the first audio segment always lesser.

      <SegmentTimeline>
        <S t="126000" d="54332"/>
        <S t="180332" d="89817" r="1"/>
        <S t="359966" d="91996"/>
        <S t="451962" d="89861" r="13"/>
        <S t="1710016" d="91954"/>
        <S t="1801970" d="89861" r="3"/>
      </SegmentTimeline> 

[kqyang]

I just realized that I made a mistake in #554 (comment): The duration of the first segment is 2 seconds instead of 20 seconds. So please ignore that comment.

@sr1990 @sridhard And yes, the duration of the first segment is expected to be smaller or equal to the duration of the later segments for live packaging as it is determined by the first frame packager receives.

@sr1990 The changes you did in #554 (comment) is in the right direction, but it needs to be modified. There are also additional changes needed. Can you submit your changes as a pull request? It will be easier to comment on your pull request. Appreciated!

[sr1990]

I just realized that I made a mistake in #554 (comment): The duration of the first segment is 2 seconds instead of 20 seconds. So please ignore that comment.

@sr1990 @sridhard And yes, the duration of the first segment is expected to be smaller or equal to the duration of the later segments for live packaging as it is determined by the first frame packager receives.

@sr1990 The changes you did in #554 (comment) is in the right direction, but it needs to be modified. There are also additional changes needed. Can you submit your changes as a pull request? It will be easier to comment on your pull request. Appreciated!

@kqyang Have created a PR.

[alwaysAn0n]

Eagerly awaiting a fix for this. Thanks for all your hard work friends :)