Significant video artifacts that appear only in Safari (and not in the default Safari HLS player)

[eloyer]

What version of Hls.js are you using?

v1.4.4

What browser (including version) are you using?

Safari 16.2 (18614.3.7.1.5)

What OS (including version) are you using?

Mac OS 13.1 (22C65)

Test stream

https://customer-vt66666ts449vzd9.cloudflarestream.com/a275a6db430d58cd6a2f778ebe44dd99/manifest/video.m3u8

Configuration

{
  capLevelToPlayerSize: false,
  startLevel: 3,
}

Additional player setup steps

No response

Checklist

• The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
• The issue occurs in the stable client (latest release) on https://hlsjs.video-dev.org/demo and not just on my page
• The issue occurs in the latest client (main branch) on https://hlsjs-dev.video-dev.org/demo and not just on my page
• The stream has correct Access-Control-Allow-Origin headers (CORS)
• There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

1. Play the video (it happens every time)

Expected behaviour

Clean video playback (screenshot is from Chrome).

What actually happened?

Video displays artifacting (screenshot is from Safari).

Console output

[Log] [log] > – "Debug logs enabled for \"Hls instance\" in hls.js version 1.4.4" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 79)
[Log] [log] > – "stopLoad" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 2893)
[Log] [log] > – "loadSource:https://customer-vt66666ts449vzd9.cloudflarestream.com/a275a6db430d58cd6a2f778ebe44dd99/manifest/video.m3u8" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 2884)
[Log] [log] > – "[stream-controller]:" – "Trigger BUFFER_RESET" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1890)
[Log] [log] > – "attachMedia" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 2880)
[Log] [log] > – "[buffer-controller]: Media source opened" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 2144)
[Log] [log] > – "[subtitle-stream-controller]:" – "STOPPED->IDLE" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[level-controller]:" – "manifest loaded, 5 level(s) found, first bitrate: 1530438" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 660)
[Log] [log] > – "1 bufferCodec event(s) expected" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 2149)
[Log] {autoStartLoad: true, startPosition: -1, defaultAudioCodec: undefined, debug: true, capLevelOnFPSDrop: false, …} (app.545bbeaa7897161a.hot-update.js, line 126)
[Log] [log] > – "startLoad(-1)" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 2891)
[Log] [log] > – "[level-controller]:" – "Switching to level 3 from level -1" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 665)
[Log] [log] > – "[level-controller]:" – "Loading level index 3 with URI 1/1 https://customer-vt66666ts449vzd9.cloudflarestream.com/a275a6db430d58cd6a2f778ebe44dd99/manifest/stream_t…" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 675)
"Loading level index 3 with URI 1/1 https://customer-vt66666ts449vzd9.cloudflarestream.com/a275a6db430d58cd6a2f778ebe44dd99/manifest/stream_t4bc7030b218129f92d08069a17d9ed8e_r419229644.m3u8"
[Log] [log] > – "[stream-controller]:" – "STOPPED->IDLE" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[subtitle-stream-controller]:" – "IDLE->STOPPED" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[subtitle-stream-controller]:" – "STOPPED->IDLE" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "Level 3 loaded [0,3][part-3--1], cc [0, 0] duration:13" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1891)
[Log] [log] > – "[buffer-controller]: Updating Media Source duration to 13.000" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 2195)
[Log] [log] > – "[stream-controller]:" – "Loading fragment 0 cc: 0 of [0-3] level: 3, target: 0" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 921)
[Log] [log] > – "[stream-controller]:" – "IDLE->FRAG_LOADING" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[transmuxer-interface, main]: Starting new transmux session for sn: 0 p: -1 level: 3 id: 1↵        discontinuity: true↵        trackSwitch…" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1780)
"[transmuxer-interface, main]: Starting new transmux session for sn: 0 p: -1 level: 3 id: 1
        discontinuity: true
        trackSwitch: true
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 0
        initSegmentChange: true"
[Log] [log] > – "[mp4-remuxer]: ISGenerated flag reset" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1517)
[Log] [log] > – "[mp4-remuxer]: initPTS & initDTS reset" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1517)
[Log] [log] > – "[mp4-remuxer]: reset next timestamp" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1517)
[Log] [log] > – "[stream-controller]:" – "FRAG_LOADING->PARSING" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "Init video buffer, container:video/mp4, codecs[level/parsed]=[avc1.4d401f/avc1.64001f]" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1936)
[Log] [log] > – "[buffer-controller]: creating sourceBuffer(video/mp4;codecs=avc1.4d401f)" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 2202)
[Log] [log] > – "[audio-stream-controller]:" – "InitPTS for cc: 0 found from main: 7500" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 2038)
[Log] [log] > – "[stream-controller]:" – "Loaded fragment 0 of level 3" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 909)
[Log] [log] > – "[transmuxer.ts]: Flushed fragment 0 of level 3" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1652)
[Log] [log] > – "[stream-controller]:" – "PARSING->PARSED" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "Buffered main sn: 0 of level 3 (frag:[0.000-4.000] > buffer:[0.000-4.000])" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 918)
[Log] [log] > – "[stream-controller]:" – "PARSED->IDLE" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "Adapting to level 4 from level 3" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1856)
[Log] [log] > – "[level-controller]:" – "Switching to level 4 from level 3" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 665)
[Log] [log] > – "[level-controller]:" – "Loading level index 4 with URI 1/1 https://customer-vt66666ts449vzd9.cloudflarestream.com/a275a6db430d58cd6a2f778ebe44dd99/manifest/stream_t…" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 675)
"Loading level index 4 with URI 1/1 https://customer-vt66666ts449vzd9.cloudflarestream.com/a275a6db430d58cd6a2f778ebe44dd99/manifest/stream_t4bc7030b218129f92d08069a17d9ed8e_r419229702.m3u8"
[Log] [log] > – "[stream-controller]:" – "IDLE->WAITING_LEVEL" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] Level switched – 3 (app.545bbeaa7897161a.hot-update.js, line 123)
[Log] [log] > – "[stream-controller]:" – "Level 4 loaded [0,3][part-3--1], cc [0, 0] duration:13" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1891)
[Log] [log] > – "[stream-controller]:" – "WAITING_LEVEL->IDLE" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "Loading fragment 1 cc: 0 of [0-3] level: 4, target: 4" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 921)
[Log] [log] > – "[stream-controller]:" – "IDLE->FRAG_LOADING" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[transmuxer-interface, main]: Starting new transmux session for sn: 1 p: -1 level: 4 id: 1↵        discontinuity: false↵      …" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1780)
"[transmuxer-interface, main]: Starting new transmux session for sn: 1 p: -1 level: 4 id: 1
        discontinuity: false
        trackSwitch: true
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 4
        initSegmentChange: false"
[Log] [log] > – "[mp4-remuxer]: ISGenerated flag reset" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1517)
[Log] [log] > – "[mp4-remuxer]: initPTS & initDTS reset" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1517)
[Log] [log] > – "[mp4-remuxer]: reset next timestamp" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1517)
[Log] [log] > – "[stream-controller]:" – "FRAG_LOADING->PARSING" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "Init video buffer, container:video/mp4, codecs[level/parsed]=[avc1.4d4028/avc1.4d4028]" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1936)
[Log] [log] > – "[stream-controller]:" – "Loaded fragment 1 of level 4" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 909)
[Log] [log] > – "[transmuxer.ts]: Flushed fragment 1 of level 4" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1652)
[Log] [log] > – "[stream-controller]:" – "PARSING->PARSED" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "Buffered main sn: 1 of level 4 (frag:[3.917-7.917] > buffer:[0.000-3.833][3.917-7.917])" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 918)
[Log] [log] > – "[stream-controller]:" – "PARSED->IDLE" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "Loading fragment 2 cc: 0 of [0-3] level: 4, target: 7.917" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 921)
[Log] [log] > – "[stream-controller]:" – "IDLE->FRAG_LOADING" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "FRAG_LOADING->PARSING" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "Loaded fragment 2 of level 4" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 909)
[Log] [log] > – "[transmuxer.ts]: Flushed fragment 2 of level 4" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1652)
[Log] [log] > – "[stream-controller]:" – "PARSING->PARSED" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "Buffered main sn: 2 of level 4 (frag:[7.917-11.917] > buffer:[0.000-3.833][3.917-11.917])" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 918)
[Log] [log] > – "[stream-controller]:" – "PARSED->IDLE" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "Loading fragment 3 cc: 0 of [0-3] level: 4, target: 11.917" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 921)
[Log] [log] > – "[stream-controller]:" – "IDLE->FRAG_LOADING" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "FRAG_LOADING->PARSING" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "Loaded fragment 3 of level 4" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 909)
[Log] [log] > – "[transmuxer.ts]: Flushed fragment 3 of level 4" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 1652)
[Log] [log] > – "[stream-controller]:" – "PARSING->PARSED" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[stream-controller]:" – "Buffered main sn: 3 of level 4 (frag:[11.917-13.000] > buffer:[0.000-3.833][3.917-13.000])" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 918)
[Log] [log] > – "[stream-controller]:" – "PARSED->IDLE" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[buffer-controller]: video sourceBuffer now EOS" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 2182)
[Log] [log] > – "[buffer-controller]: Queueing mediaSource.endOfStream()" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 2182)
[Log] [log] > – "[stream-controller]:" – "IDLE->ENDED" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 995)
[Log] [log] > – "[buffer-controller]: Calling mediaSource.endOfStream()" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 2182)
[Log] [log] > – "[buffer-controller]: Media source ended" (chunk-vendors.545bbeaa7897161a.hot-update.js, line 2145)

Chrome media internals output

No response

[robwalch]

Thanks for the bug report @eloyer. It looks like these artifacts were introduced with a change in v1.2.2. Prior to that change we could still see a buffer gap between these segments in Safari going back to v0.14. In v0.12.1 we can see no gap, but jittering between frames. So long story short, HLS.js has always had issues with M2TS segments like this (in Safari).

If we use startPosition of 5 to skip the first segment, the artifacts are gone, but they come back if we seek back to 0. The main difference being that with an initPTS of 0, appends work correctly. With in initPTS of 7500, the appends are shifted back, likely messing with composition time of the initial frames. The larger PTS offset in an attempt to eliminate start gaps, but this results in decoding artifacts (in Safari).

In the logs I also see overlap between fragments (related to the difference in starting PTS offset noticed above):

[Warning] [warn] > – "AVC: 83 ms (-7500dts) overlapping between fragments detected" (hls.js, line 14968)

I'll need to take a closer look at the AVC samples and their presentation and decode timestamps to get a better idea of what is going on here. Just a guess, but this user-agent probably requires appends based on decode timestamps rather than presentation timestamps. At the start of a VOD stream, if HLS.js avoids setting an initPTS that could result in negative decode timestamps, while we might end up with a start gap, playback should be OK.

[eloyer]

Thanks for the quick response — can confirm that downgrading to 1.2.1 resolves the issue, this works for now until a fix is available.