-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(mp4-tools): Fix two AV1 parsing issues #5774
Conversation
Thank you for the contribution @nyanmisaka! I'll review soon with the hopes of including these fixes in the next release. |
src/utils/mp4-tools.ts
Outdated
const monochrome = (av1CBox[2] & 0x10) >> 4; | ||
const chromaSubsamplingX = (av1CBox[2] & 0x08) >> 3; | ||
const chromaSubsamplingY = (av1CBox[2] & 0x04) >> 2; | ||
const chromaSamplePosition = av1CBox[2] & 0x03; | ||
codec += | ||
'.' + | ||
profile + | ||
'.' + | ||
addLeadingZero(level) + | ||
tierFlag + | ||
'.' + | ||
addLeadingZero(bitDepth) + | ||
'.' + | ||
monochrome + | ||
'.' + | ||
chromaSubsamplingX + | ||
chromaSubsamplingY + | ||
chromaSamplePosition; | ||
addLeadingZero(bitDepth); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am concerned that some user-agents may still depend these fields being present.
https://aomediacodec.github.io/av1-isobmff/#av1codecconfigurationbox-syntax
The parameters sample entry 4CC, profile, level, tier, and bitDepth are all mandatory fields. If any of these fields are empty, or not within their allowed range, the processing device SHOULD treat it as an error. All the other fields (including their leading '.') are optional, mutually inclusive (all or none) fields. If not specified then the values listed in the table below are assumed.
mono_chrome 0 chromaSubsampling 110 (4:2:0) colorPrimaries 01 (ITU-R BT.709) transferCharacteristics 01 (ITU-R BT.709) matrixCoefficients 01 (ITU-R BT.709) videoFullRangeFlag 0 (studio swing representation)
Would adding the default values defined above work for Firefox?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @nyanmisaka,
Eventually (not in this PR), we may need a workaround for incomplete AV1 codec strings in Multivariant Playlist CODECS sort of like the workaround for opus/flac strings:
Lines 132 to 172 in 6227816
function getCodecCompatibleNameLower( | |
lowerCaseCodec: LowerCaseCodecType, | |
preferManagedMediaSource = true, | |
): string { | |
if (CODEC_COMPATIBLE_NAMES[lowerCaseCodec]) { | |
return CODEC_COMPATIBLE_NAMES[lowerCaseCodec]!; | |
} | |
// Idealy fLaC and Opus would be first (spec-compliant) but | |
// some browsers will report that fLaC is supported then fail. | |
// see: https://bugs.chromium.org/p/chromium/issues/detail?id=1422728 | |
const codecsToCheck = { | |
flac: ['flac', 'fLaC', 'FLAC'], | |
opus: ['opus', 'Opus'], | |
}[lowerCaseCodec]; | |
for (let i = 0; i < codecsToCheck.length; i++) { | |
if ( | |
isCodecMediaSourceSupported( | |
codecsToCheck[i], | |
'audio', | |
preferManagedMediaSource, | |
) | |
) { | |
CODEC_COMPATIBLE_NAMES[lowerCaseCodec] = codecsToCheck[i]; | |
return codecsToCheck[i]; | |
} | |
} | |
return lowerCaseCodec; | |
} | |
const AUDIO_CODEC_REGEXP = /flac|opus/i; | |
export function getCodecCompatibleName( | |
codec: string, | |
preferManagedMediaSource = true, | |
): string { | |
return codec.replace(AUDIO_CODEC_REGEXP, (m) => | |
getCodecCompatibleNameLower( | |
m.toLowerCase() as LowerCaseCodecType, | |
preferManagedMediaSource, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed. Currently AV1 is not popular yet in HLS.
But as Apple provides native support for AV1, this problem will become apparent.
def894d
to
05737e6
Compare
src/utils/mp4-tools.ts
Outdated
const monochrome = (av1CBox[2] & 0x10) >> 4; | ||
const chromaSubsamplingX = (av1CBox[2] & 0x08) >> 3; | ||
const chromaSubsamplingY = (av1CBox[2] & 0x04) >> 2; | ||
const chromaSamplePosition = av1CBox[2] & 0x03; | ||
codec += | ||
'.' + | ||
profile + | ||
'.' + | ||
addLeadingZero(level) + | ||
tierFlag + | ||
'.' + | ||
addLeadingZero(bitDepth) + | ||
'.' + | ||
monochrome + | ||
'.' + | ||
chromaSubsamplingX + | ||
chromaSubsamplingY + | ||
chromaSamplePosition; | ||
addLeadingZero(bitDepth); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @nyanmisaka,
Eventually (not in this PR), we may need a workaround for incomplete AV1 codec strings in Multivariant Playlist CODECS sort of like the workaround for opus/flac strings:
Lines 132 to 172 in 6227816
function getCodecCompatibleNameLower( | |
lowerCaseCodec: LowerCaseCodecType, | |
preferManagedMediaSource = true, | |
): string { | |
if (CODEC_COMPATIBLE_NAMES[lowerCaseCodec]) { | |
return CODEC_COMPATIBLE_NAMES[lowerCaseCodec]!; | |
} | |
// Idealy fLaC and Opus would be first (spec-compliant) but | |
// some browsers will report that fLaC is supported then fail. | |
// see: https://bugs.chromium.org/p/chromium/issues/detail?id=1422728 | |
const codecsToCheck = { | |
flac: ['flac', 'fLaC', 'FLAC'], | |
opus: ['opus', 'Opus'], | |
}[lowerCaseCodec]; | |
for (let i = 0; i < codecsToCheck.length; i++) { | |
if ( | |
isCodecMediaSourceSupported( | |
codecsToCheck[i], | |
'audio', | |
preferManagedMediaSource, | |
) | |
) { | |
CODEC_COMPATIBLE_NAMES[lowerCaseCodec] = codecsToCheck[i]; | |
return codecsToCheck[i]; | |
} | |
} | |
return lowerCaseCodec; | |
} | |
const AUDIO_CODEC_REGEXP = /flac|opus/i; | |
export function getCodecCompatibleName( | |
codec: string, | |
preferManagedMediaSource = true, | |
): string { | |
return codec.replace(AUDIO_CODEC_REGEXP, (m) => | |
getCodecCompatibleNameLower( | |
m.toLowerCase() as LowerCaseCodecType, | |
preferManagedMediaSource, |
Profile: unsigned right shift 5-bits for the 2nd av1CBox Level: bitwise AND the 2nd av1CBox with 0b11111(0x1f) Without this, Main 'av01.0.15M.08' is parsed as High 'av01.1.15M.08'. In this case the High profile is not supprted in Firefox. Signed-off-by: nyanmisaka <nst799610810@gmail.com>
In Firefox either you omit the chroma values, or you have to complete the rest of the color descriptions. Otherwise even the most basic AV1 Main 4:2:0 8-bit video will be reported as unsupported in HTMLMediaElement.canPlayType and MediaSource.isTypeSupported. Chromium is not affected by this. Signed-off-by: nyanmisaka <nst799610810@gmail.com>
05737e6
to
0d4cfde
Compare
# 1.0.0 (2025-02-26) * [JW8-6113] Implement progressive AES decryption ([video-dev#219](https://github.com/teiron-1604/hls.js/issues/219)) ([1050e74](1050e74)) ### Bug Fixes * `setMediaKeys` return undefined in some Chrome versions ([video-dev#6669](https://github.com/teiron-1604/hls.js/issues/6669)) ([1d10714](1d10714)) * abrFactor greater than in docs ([1adc39e](1adc39e)) * afterEach callback should return a promise ([44aa769](44aa769)) * baseMediaDecodeTime in tfdt v0 should not be re-written to a negative value ([b884798](b884798)) * calling detatchMedia() followed by attachMedia() causes audio to not play ([161c66c](161c66c)), closes [video-dev#2099](https://github.com/teiron-1604/hls.js/issues/2099) * catch exception ([715ca74](715ca74)) * change test streams hosting to mux ([video-dev#2307](https://github.com/teiron-1604/hls.js/issues/2307)) ([25f0bbe](25f0bbe)) * **ci:** invalid branch ([5c20968](5c20968)) * computeLivePosition minimum value of media.currentTime ([3fd87fe](3fd87fe)) * consider paused state ([b58c595](b58c595)), closes [video-dev#2417](https://github.com/teiron-1604/hls.js/issues/2417) * contiguous detection error ([2a9180d](2a9180d)) * Delete unused var ([9d3beb9](9d3beb9)) * **demo:** update http to https ([9327fc6](9327fc6)) * **deps:** update to esdoc@1.1.0 ([ad6a9e6](ad6a9e6)) * **docs:** fix links to src ([b82e921](b82e921)) * duplicate segmented vtt lines ([a25f0b1](a25f0b1)) * failed to switch back to main audio from additional audio ([video-dev#5170](https://github.com/teiron-1604/hls.js/issues/5170)) ([6e157ac](6e157ac)) * Fix indent ([6eb6eaf](6eb6eaf)) * fix live playlist not sync ([938606f](938606f)) * **func-test:** add "firefox" to black_list of angelOneShakaWidevine ([cb6e22e](cb6e22e)) * **func-test:** add `nudgeMaxRetry` option for slow seek env ([aca699c](aca699c)) * generate silent aac frame based on original codec ([video-dev#6123](https://github.com/teiron-1604/hls.js/issues/6123)) ([b721af3](b721af3)) * initial segment change without discontinuity ([ab9651e](ab9651e)) * **latency-controller:** only sync live stream ([d3845c2](d3845c2)) * level playlist parse error due to "rawProgramDateTime" of initial segment not reset ([a4b4f5f](a4b4f5f)) * lost program date time. ([7fef02c](7fef02c)) * move `types` condition to the front ([video-dev#5439](https://github.com/teiron-1604/hls.js/issues/5439)) ([9082342](9082342)) * **mp4-tools:** Fix two AV1 parsing issues ([video-dev#5774](https://github.com/teiron-1604/hls.js/issues/5774)) ([c9e9916](c9e9916)) * Not set initial segment for part and hint segments ([3331d6f](3331d6f)) * partial audiovideo fragments not being treated as partial ([video-dev#5460](https://github.com/teiron-1604/hls.js/issues/5460)) ([ef718d2](ef718d2)) * remove redundant code ([d8d684d](d8d684d)) * remove uneeded tests ([95f32a1](95f32a1)) * Remove unnecessary option ([video-dev#2273](https://github.com/teiron-1604/hls.js/issues/2273)) ([f723c06](f723c06)) * **Remuxer:** Safari segment overlap ensure PTS order ([video-dev#6132](https://github.com/teiron-1604/hls.js/issues/6132)) ([af42912](af42912)) * Replace Loader.getResponseHeader() with Loader.getCacheAge(). ([d520008](d520008)) * Replace tab to space ([6eec3df](6eec3df)) * set initial segment repeatedly for ll-hls stream ([78de428](78de428)) * show cc with low latency stream. ([6e23bd0](6e23bd0)) * stopLoad() after destroy() becomes no-op, startLoad() after destroy() throws nicer error message. ([0c32415](0c32415)) * StreamController#stop should call clearInterval ([1aa78bf](1aa78bf)) * **subtitle:** vtt fragment sn is consistent with ts, no need to request previous fragment sn ([6d1f3db](6d1f3db)) * the functional tests for "MPEG Audio Only demo" failed ([732c2c3](732c2c3)) * **timeline-controller:** reuse text tracks when not in same order ([video-dev#1869](https://github.com/teiron-1604/hls.js/issues/1869)) ([6db0112](6db0112)) * tizen 2017 widevine playback ([video-dev#6657](https://github.com/teiron-1604/hls.js/issues/6657)) ([f167465](f167465)) * type name error ([video-dev#5295](https://github.com/teiron-1604/hls.js/issues/5295)) ([91364bb](91364bb)) * **types:** Remove unused Loader.loader property. ([fb1ead0](fb1ead0)) * **type:** use extended generic type `LoaderContext` in type `Loader` ([video-dev#5487](https://github.com/teiron-1604/hls.js/issues/5487)) ([075e834](075e834)) * typo ([f769aac](f769aac)) * unit test fail, observer.emit is not a function ([6b85ffa](6b85ffa)) * use hasInterval instead of isNaN ([a9e6c45](a9e6c45)) * using implicit return ([4bf6ee9](4bf6ee9)) * vtt parser broke after master merge ([e96a2a1](e96a2a1)) * X-EXT-MAP in between duration and url ([d7e90c6](d7e90c6)) * X-EXT-MAP in between duration and url ([078fddc](078fddc)) ### Code Refactoring * Remove loader.getResponseHeader(). ([ca962e5](ca962e5)) ### Features * add hasFragment and removeAllFragments ([3ae3aa3](3ae3aa3)) * add support for CMCD nor ([video-dev#6091](https://github.com/teiron-1604/hls.js/issues/6091)) ([763c129](763c129)), closes [video-dev#6088](https://github.com/teiron-1604/hls.js/issues/6088) * add support for decrypting init segments. ([41addd9](41addd9)), closes [video-dev#2259](https://github.com/teiron-1604/hls.js/issues/2259) * Add supported M3U8 tags ([2ca6bb8](2ca6bb8)) * add type to discontinuities.js and fix @typescript-eslint/prefer-optional-chain error ([ad1bbef](ad1bbef)) * **ci:** add release workflow ([154409c](154409c)) * Format code by Prettier ([993ef82](993ef82)) * **level:** deliver level details when level switches ([55e4dbc](55e4dbc)) * using ts assert instead of runtime code changes ([0674273](0674273)) ### Performance Improvements * for .. of to simple for iteration ([409a1ab](409a1ab)) * inlining bufferedIncludesPosition ([ad5bfa1](ad5bfa1)) * pre-cache buffered positions ([03aa7ec](03aa7ec)) * use helper function to filter evicted fragments ([34190c0](34190c0)) ### Reverts * Revert "buffer-helper: correctly report bufferInfo when current time is located before buffered ranges, but within maxHoleDuration range" ([877f1c6](877f1c6)), closes [video-dev#420](https://github.com/teiron-1604/hls.js/issues/420) * Revert "config defaults: set maxBufferHole to 15 second" ([8acc166](8acc166)) * Revert "Configure rennovate to `pinGitHubActionDigests`" ([2c993b1](2c993b1)) * Revert "Disable `AppleAdvancedHevcAvcHls` stream for now due to cors errors (…" ([video-dev#6187](https://github.com/teiron-1604/hls.js/issues/6187)) ([efb252c](efb252c)) * Revert "functional test: run against Safari 10.0" ([be5d7c5](be5d7c5)) * Revert "switch to npm 7 ([video-dev#3666](https://github.com/teiron-1604/hls.js/issues/3666))" ([video-dev#3671](https://github.com/teiron-1604/hls.js/issues/3671)) ([2e3f6f3](2e3f6f3)) ### BREAKING CHANGES * Remove loader.getResponseHeader() in favor of loader.getCacheAge(). * force software AES decrypter if config is true
This PR will...
Fix two AV1 parsing issues that make Firefox fails to play HLS stream
Why is this Pull Request needed?
fix(mp4-tools): Fix AV1 profile parsing
Profile: unsigned right shift 5-bits for the 2nd av1CBox
Level: bitwise AND the 2nd av1CBox with 0b11111(0x1f)
Without this, Main
av01.0.15M.08
is parsed as Highav01.1.15M.08
.In this case the High profile is not supprted in Firefox.
fix(mp4-tools): set default color descriptions to satisfy Firefox
In Firefox either you omit the chroma values, or you have to complete the rest of the color descriptions.
Otherwise even the most basic AV1 Main 4:2:0 8-bit video will be reported as unsupported in HTMLMediaElement.canPlayType and MediaSource.isTypeSupported. Chromium is not affected by this.
Are there any points in the code the reviewer needs to double check?
See also
Resolves issues:
av1 Main is mistakenly parsed as High profile and eventually fails in Firefox


playlist:
log:
Firefox is unhapply with the chroma values

Checklist