[gpu-next] HDR metadata not forwarded in windowed mode

[mitzsch]

Important Information

Provide following Information:

• mpv version: mpv-dev-x86_64-v3-20220909-git-d433c5d
• Windows Version: Windows 10 21H2
• Source of the mpv binary: shinchiro winbuild repo
• Hardware: AMD RX580 / Nvidia RTX3050

Reproduction steps

I recently purchased a hdfury arcana to debug HDMI on my PC and other devices. I wanted to know if and which HDR metadata values mpv passes through to the TV. With mpv being in fullscreen, HDR turned on in Windows, and the following command line
mpv.com HDR.file -vo=gpu-next -hwdec=d3d11va -target-colorspace-hint=yes -v -fs
it works perfectly, as seen in the picture.

However when running mpv with the same command just without -fs and therefore in windowed mode, no metadata is passed through. The hdfury shows the default windows HDR values. (For nvidia all 0, for AMD 1499maxCLL/799maxFALL)
This behavior is quite unfortunate, as any software that embeds mpv and is not able to run it as fullscreen is out of luck passing through actual metadata. Also for daily use, where mpv should just run as a separate window it is unfortunate. It would be awesome if it could be added, either out of the box or with a dedicated allow command.

Expected behavior

Allow for metadata passthrough even in windowed mode.

Actual behavior

No metadata forwarded when running mpv in windowed mode.

Sample files

Any HDR file with metadata present

[mitzsch]

Thanks for your answer! I tried it with Vulkan on the RTX 3050 system. (On the AMD system it would not work at all with Vulkan) With the RTX 3050 the TV switched to HDR but received no metadata at all. (not in windowed nor in fullscreen mode) The hdfury showed 0 for all values. The only way I was able to get passthrough with Vulkan was with activated HDR under windows settings and even then it only worked in fullscreen. (note: information regarding colorspace were not transmitted with Vulkan - the hdfury showed unknown for target colorspace)

[mitzsch]

update

This seems to be a Microsoft / Windows problem. The sample code called Direct3D 12 HDR sample shows the same behavior (metadata only forwarded in fullscreen mode), so it's not solely related to mpv.

microsoft/DirectX-Graphics-Samples#797

[mitzsch]

Valid point but in case where other programs are on the same screen (or the wallpaper), the windows compositor should handle the mapping.

[mitzsch]

Sure!
I did some extensive testing with different settings on both the Nvidia RTX3050 and the AMD RX580 machine. (Both running Windows 10 21H2)

Used command:
mpv.com HDR.mkv -vo=gpu-next -hwdec=d3d11va -d3d11-exclusive-fs=yes/no -target-colorspace-hint=yes -v

Here are the results:

AMD

• fullscreen optimization disabled, --d3d11-exclusive-fs=no
  => Passthrough only in fullscreen
  => Metadata not flushed when quitting in windowed mode
  => Metadata correctly flushed when quitting (or file ends) in fullscreen mode

• fullscreen optimization disabled, --d3d11-exclusive-fs=yes
  => when in fullscreen, HDR passthrough is completely broken, the TV even switches out of HDR, Image looks totally wrong

• fullscreen optimization enabled, --d3d11-exclusive-fs=no
  => Passthrough only in fullscreen
  => Metadata not flushed when quitting in windowed mode
  => Metadata correctly flushed when quitting (or file ends) in fullscreen mode => also when using --idle=yes

• fullscreen optimization enabled, --d3d11-exclusive-fs=yes
  => Passthrough only in fullscreen
  => Metadata correctly flushed when quitting in fullscreen or windowed mode (or file ends) => also when using --idle=yes

---

Nvidia

• fullscreen optimization disabled, --d3d11-exclusive-fs=no
  => Passthrough only in fullscreen
  => Metadata not flushed when quitting in windowed mode
  => When exiting fullscreen colorspace metadata is reset (brightness values are still in the hdmi stream)
  => Metadata correctly flushed when quitting (or file ends) in fullscreen mode

• fullscreen optimization disabled, --d3d11-exclusive-fs=yes
  => when in fullscreen, HDR passthrough is completely broken, the TV even switches out of HDR, Image looks totally wrong

• fullscreen optimization enabled, --d3d11-exclusive-fs=no
  => Passthrough only in fullscreen
  => Metadata not flushed when quitting in windowed or fullscreen mode

• fullscreen optimization enabled, --d3d11-exclusive-fs=yes
  => Passthrough only in fullscreen
  => Metadata correctly flushed when quitting in fullscreen or windowed mode (or file ends)
  => When using it with --idle=yes metadata is not flushed after file ends

---

Additional Note:

Nvidia does some weird trickery. When HDR is enabled in the windows setting no brightness values are transmitted. When watching HDR files (especially the file used for testing here "03. 900-4000nits-MaxCLL-4000-MDL-4000.mp4" from avforums) the results seem fine even though no actual metadata is transmitted. (at least not visible for the hdfury arcana - I guess its true that there is no metadata in the stream - the tech support of hdfury told me that the arcana grabs everything on the HDMI link). On the AMD side, you can cleary see, when actual metadata is present. The "03. 900-4000nits-MaxCLL-4000-MDL-4000.mp4" file is clipped above 1500 nits (the default maxCLL value is 1499 for AMD) when watching with an AMD card and no metadata is present.

Small explanation:
I used the word flushed for the state when the brightness values are reset to their "windows default". In some cases, the metadata is stuck in the stream (for example when exiting fullscreen mode) and won´t get reset (even after mpv is closed). So for example a maxCLL of 4000nit is still present in the HDMI stream even though mpv has been closed.

[mitzsch]

You need to select 10 bit in nvidia control panel.

Well, 10bit is not a necessity for HDR. Nor on the AMD or the Nvidia side of things. I just tried it with 10bit on both systems and the result is the same. Using HDR with 8bit will introduce banding, but that's not what I meant to say. On both machines running fullscreen optimization disabled, --d3d11-exclusive-fs=yes , the TV will switch out of HDR back to SDR and therefore it's not surprising that the results will look off. (I should have been more precise on that one - sorry)

[Nevcairiel]

As far as I am aware, this is quite simply a Windows limitation. You don't need fullscreen exclusive mode, you just need a window that fully covers the entire screen perfectly. Only then will the Windows compositor let you take over full control of the output and use your HDR metadata, rather then its default values.

Allowing smaller windows to do the same invites several issues. What if multiple windows show HDR content, which metadata is favored? Hence why I assume this is not supported. I doubt you'll find any app that behaves differently here, unless they find a way to go behind Windows' back somehow, which probably shouldn't be the way to go.