[Remote Playback] Capabilities for HDR rendering and display

[markafoltz]

This issue is tracking requirements to allow one agent to determine another's capabilities to render and display HDR media, both for streaming and remote playback.

At the Berlin F2F [1], it was noted that HDR capabilities are under discussion in the Media WG, and this issue is tracking the work being done there to align the protocol with the data model that will be used:

• Add Support for Querying HDR Decode and Render Capabilities media-capabilities#118: Add Support for Querying HDR Decode and Render Capabilities
• Add Support for Querying HDR & 4K Video Display Capabilities media-capabilities#119: Add Support for Querying HDR & 4K Video Display Capabilities

It looks likely that this topic will be on the agenda of the Media WG at TPAC 2019.

[markafoltz]

I'm marking this as v1-spec in the hopes that it will be resolved in the Media WG in a timely manner :-)

[markafoltz]

Ref: https://www.w3.org/2019/09/15-webscreens-minutes.html#x16

w3c/media-capabilities#118 has been closed, but w3c/media-capabilities#119 remains open. I think it would make sense to land a PR to align with #118.

[anssiko]

ACTION: @mfoltzgoogle to propose a PR to align OSP with Media Capabilities issue w3c/media-capabilities#118

via https://www.w3.org/2020/10/20-webscreens-minutes.html#a04

[anssiko]

w3c/media-capabilities#119 was closed and spawned two issues:

• Screen.video.hdrSupported hdrSupported - Screen.video or simply Screen? media-capabilities#135
• CSS length units in video-* media features [mediaqueries] length units in video-* media features. csswg-drafts#5044

[markafoltz]

It sounds like there wasn't an explicit action recorded in the last F2F minutes [1], but for the record, we discussed consulting with the Media WG folks working on Media Capabilities and related APIs to get feedback on PR #225, to resolve the first of these two issues (the render/decoding capabilities).

[1] https://www.w3.org/2021/02/23-webscreens-minutes.html#t02

[chrisn]

Please let me know if you'd like to bring this to a Media WG meeting, happy to put this on the agenda.

[anssiko]

@chrisn given w3c/media-capabilities#135 and w3c/csswg-drafts#5044 haven't received attention recently I'd appreciate that. Please drop meeting logistics pointer to this issue so interested people can join.

[chrisn]

Meeting details are linked here. Upcoming dates are Tuesday 8 June at 14:00 UTC, 6 July at 21:00 UTC. Do you have any preference for date?

/cc @jernoble

[anssiko]

@mfoltzgoogle please note the upcoming Media WG meeting dates where this issue and its collateral are discussed. I think you may want to participate, and I suspect the latter 6 July at 21:00 UTC would be preferred by you (I may not be available at that time, but don't block on me).

[markafoltz]

Didn't see this before the June 8 meeting (sorry!) and July 6 is a Google holiday here in the US. However I'll follow whatever the agenda the Media WG wishes on this item - whether that's July, August or later.

[chrisn]

Here are the upcoming Media WG dates:

• 9 November 22:00 UTC / 14:00 US Pacific
• 14 December 15:00 UTC / 07:00 US Pacific
• 11 January 22:00 UTC / 14:00 US Pacific

Please let me know your preference, @mfoltzgoogle @anssiko @tidoust

[markafoltz]

My preference order:

• 11 January
• 14 December
• 9 November

[anssiko]

I will probably only be able to join on 14 Dec, but will defer to @mfoltzgoogle preferences who's on the critical path for these issues. Also adding @louaybassbouss

[louaybassbouss]

I will probably only be able to join on 14 Dec, but will defer to @mfoltzgoogle preferences who's on the critical path for these issues. Also adding @louaybassbouss

same for me 14 December is the preferred date but 11 January works as well.

[chrisn]

Let's arrange for 11 January, given Mark's preferences.

[chrisn]

The joint meeting with the Media WG is confirmed for 11 January, here are the meeting details.

[chrisn]

The minutes from the joint meeting with Media WG are here. Summary:

• We can refer to definitions in https://www.w3.org/TR/mediaqueries-5/ (display capabilities) and https://w3c.github.io/media-capabilities/#enumdef-colorgamut (decode capabilities)

• The definition for video plane resolution (as opposed to graphics plane resolution), for TV devices has not been settled in CSSWG. The latest issue is [cssom-view] Expose video plane pixel ratio csswg-drafts#6891

• Next step is to update PR Reference Media Capabilities ColorGamut for color space capability #225 and @chcunningham to offered to review

[markafoltz]

Relaying this comment from #225 so it can be addressed in a future PR:

For MediaCapabilities, colorGamut is just one of 3 properties needed to support for a given flavor of HDR. The others are TransferFunction and HdrMetadataType. Do those make sense as additional parameters in this spec?

Yes they would, if they affect how the sender might want to encode media (or decide if a media stream was compatible with a receiver). I'd like to tackle that in a separate PR since conveying the HDR data over the wire could require additional changes to the protocol.

[markafoltz]

Would like to propose landing the PR #225 to address the specification of colorGamut and opening another issue to specify the other two pieces of metadata needed for HDR media transmission (TransferFunction and HdrMetadataType).

[markafoltz]

PR #225 was already landed (missed that earlier in this thread - oops.)

Since this GitHub issue is already referenced in the spec as Issue #16, rather than open a new issue, let's continue to use this one to cover adding TransferFunction and HdrMetadataType. I believe the definitions in MediaCapabilities should be the default choice here, but need to verify the enum values correspond to the right attributes of HDR encoded media that would be sent over the wire.

From Media Capabilities:

• TransferFunction
• HdrMetadataType

[markafoltz]

With PR #300 this can be closed.

I have opened up a followup issue #306 for discussion of handling Dolby Vision formats.