Missing 4:2:2/4:4:4 DXVA hwaccel support on Intel graphics #547

nyanmisaka · 2023-07-03T15:41:34Z

First of all I want to thank you for this great project, I'm enjoying it so much on my Windows machine.

Intel graphics have supported decoding of HEVC 4:2:2 and 4:4:4 8/10bits since Ice Lake released in 2019. But even in 2023, Microsoft has not provided any official header file and struct for the HEVC Rext. Fortunately Intel made it possible for Chromium browser and a few video players such as VLC and MPC-BE to leverage it by using the vendor-specific DXVA entries.

It would be great if we can also have this feature in LAVFilters, given that 4:2:2 is becoming more and more common in the output format of high-end SLR cameras. It's difficult for users to play them on Windows smoothly without a powerful CPU, especially when it comes to 8k 4:2:2 clips.

I read this discussion from three years ago, can we revisit the patch series? Friendly paging @Nevcairiel
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200313102354.2500-1-robux4@ycbcr.xyz/

Nevcairiel · 2023-07-04T17:29:57Z

If the spec becomes official, I'll gladly support it. Until then, not adding code without a spec reference.

nyanmisaka · 2023-07-04T18:20:41Z

Intel media developer recently pointed out in Chromium-Review that Microsoft will soon release the DXVA Rext specification, we just have to wait.

Nice to see that. They almost forgot about it.

Seems like my tiny effort is paying off :P intel-media-ci/ffmpeg#602 (comment)

nyanmisaka · 2023-07-04T18:34:51Z

Intel also supports HEVC SCC, but not sure if Microsoft will also add support for this profile, a certain Intel media developer told me that at least for now there are no plans.

I don't see an urgent need for HEVC SCC hwaccel, and it wouldn't be a problem if it came later.

nyanmisaka · 2023-07-04T18:38:26Z

If the spec becomes official, I'll gladly support it. Until then, not adding code without a spec reference.

Thanks for the response!

Andarwinux · 2023-12-14T11:08:41Z

Microsoft has released the DXVA Rext specification. The relevant GUIDs can be found in the latest Canary SDK.

DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MONOCHROME, 0x0685b993, 0x3d8c, 0x43a0, 0x8b, 0x28, 0xd7, 0x4c, 0x2d, 0x68, 0x99, 0xa4);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MONOCHROME10, 0x142a1d0f, 0x69dd, 0x4ec9, 0x85, 0x91, 0xb1, 0x2f, 0xfc, 0xb9, 0x1a, 0x29);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12, 0x1a72925f, 0x0c2c, 0x4f15, 0x96, 0xfb, 0xb1, 0x7d, 0x14, 0x73, 0x60, 0x3f);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_422, 0x0bac4fe5, 0x1532, 0x4429, 0xa8, 0x54, 0xf8, 0x4d, 0xe0, 0x49, 0x53, 0xdb);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12_422, 0x55bcac81, 0xf311, 0x4093, 0xa7, 0xd0, 0x1c, 0xbc, 0x0b, 0x84, 0x9b, 0xee);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN_444, 0x4008018f, 0xf537, 0x4b36, 0x98, 0xcf, 0x61, 0xaf, 0x8a, 0x2c, 0x1a, 0x33);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_EXT, 0x9cc55490, 0xe37c, 0x4932, 0x86, 0x84, 0x49, 0x20, 0xf9, 0xf6, 0x40, 0x9c);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_444, 0x0dabeffa, 0x4458, 0x4602, 0xbc, 0x03, 0x07, 0x95, 0x65, 0x9d, 0x61, 0x7c);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12_444, 0x9798634d, 0xfe9d, 0x48e5, 0xb4, 0xda, 0xdb, 0xec, 0x45, 0xb3, 0xdf, 0x01);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN16, 0xa4fbdbb0, 0xa113, 0x482b, 0xa2, 0x32, 0x63, 0x5c, 0xc0, 0x69, 0x7f, 0x6d);

DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MONOCHROME,   0x0685b993, 0x3d8c, 0x43a0, 0x8b, 0x28, 0xd7, 0x4c, 0x2d, 0x68, 0x99, 0xa4);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MONOCHROME10, 0x142a1d0f, 0x69dd, 0x4ec9, 0x85, 0x91, 0xb1, 0x2f, 0xfc, 0xb9, 0x1a, 0x29);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12,     0x1a72925f, 0x0c2c, 0x4f15, 0x96, 0xfb, 0xb1, 0x7d, 0x14, 0x73, 0x60, 0x3f);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_422, 0x0bac4fe5, 0x1532, 0x4429, 0xa8, 0x54, 0xf8, 0x4d, 0xe0, 0x49, 0x53, 0xdb);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12_422, 0x55bcac81, 0xf311, 0x4093, 0xa7, 0xd0, 0x1c, 0xbc, 0x0b, 0x84, 0x9b, 0xee);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN_444,   0x4008018f, 0xf537, 0x4b36, 0x98, 0xcf, 0x61, 0xaf, 0x8a, 0x2c, 0x1a, 0x33);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_EXT, 0x9cc55490, 0xe37c, 0x4932, 0x86, 0x84, 0x49, 0x20, 0xf9, 0xf6, 0x40, 0x9c);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_444, 0x0dabeffa, 0x4458, 0x4602, 0xbc, 0x03, 0x07, 0x95, 0x65, 0x9d, 0x61, 0x7c);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12_444, 0x9798634d, 0xfe9d, 0x48e5, 0xb4, 0xda, 0xdb, 0xec, 0x45, 0xb3, 0xdf, 0x01);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN16,     0xa4fbdbb0, 0xa113, 0x482b, 0xa2, 0x32, 0x63, 0x5c, 0xc0, 0x69, 0x7f, 0x6d);

nyanmisaka · 2023-12-14T11:27:05Z

Microsoft has released the DXVA Rext specification. The relevant GUIDs can be found in the latest Canary SDK.

DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MONOCHROME, 0x0685b993, 0x3d8c, 0x43a0, 0x8b, 0x28, 0xd7, 0x4c, 0x2d, 0x68, 0x99, 0xa4);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MONOCHROME10, 0x142a1d0f, 0x69dd, 0x4ec9, 0x85, 0x91, 0xb1, 0x2f, 0xfc, 0xb9, 0x1a, 0x29);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12, 0x1a72925f, 0x0c2c, 0x4f15, 0x96, 0xfb, 0xb1, 0x7d, 0x14, 0x73, 0x60, 0x3f);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_422, 0x0bac4fe5, 0x1532, 0x4429, 0xa8, 0x54, 0xf8, 0x4d, 0xe0, 0x49, 0x53, 0xdb);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12_422, 0x55bcac81, 0xf311, 0x4093, 0xa7, 0xd0, 0x1c, 0xbc, 0x0b, 0x84, 0x9b, 0xee);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN_444, 0x4008018f, 0xf537, 0x4b36, 0x98, 0xcf, 0x61, 0xaf, 0x8a, 0x2c, 0x1a, 0x33);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_EXT, 0x9cc55490, 0xe37c, 0x4932, 0x86, 0x84, 0x49, 0x20, 0xf9, 0xf6, 0x40, 0x9c);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_444, 0x0dabeffa, 0x4458, 0x4602, 0xbc, 0x03, 0x07, 0x95, 0x65, 0x9d, 0x61, 0x7c);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12_444, 0x9798634d, 0xfe9d, 0x48e5, 0xb4, 0xda, 0xdb, 0xec, 0x45, 0xb3, 0xdf, 0x01);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN16, 0xa4fbdbb0, 0xa113, 0x482b, 0xa2, 0x32, 0x63, 0x5c, 0xc0, 0x69, 0x7f, 0x6d);

DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MONOCHROME,   0x0685b993, 0x3d8c, 0x43a0, 0x8b, 0x28, 0xd7, 0x4c, 0x2d, 0x68, 0x99, 0xa4);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MONOCHROME10, 0x142a1d0f, 0x69dd, 0x4ec9, 0x85, 0x91, 0xb1, 0x2f, 0xfc, 0xb9, 0x1a, 0x29);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12,     0x1a72925f, 0x0c2c, 0x4f15, 0x96, 0xfb, 0xb1, 0x7d, 0x14, 0x73, 0x60, 0x3f);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_422, 0x0bac4fe5, 0x1532, 0x4429, 0xa8, 0x54, 0xf8, 0x4d, 0xe0, 0x49, 0x53, 0xdb);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12_422, 0x55bcac81, 0xf311, 0x4093, 0xa7, 0xd0, 0x1c, 0xbc, 0x0b, 0x84, 0x9b, 0xee);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN_444,   0x4008018f, 0xf537, 0x4b36, 0x98, 0xcf, 0x61, 0xaf, 0x8a, 0x2c, 0x1a, 0x33);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_EXT, 0x9cc55490, 0xe37c, 0x4932, 0x86, 0x84, 0x49, 0x20, 0xf9, 0xf6, 0x40, 0x9c);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_444, 0x0dabeffa, 0x4458, 0x4602, 0xbc, 0x03, 0x07, 0x95, 0x65, 0x9d, 0x61, 0x7c);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12_444, 0x9798634d, 0xfe9d, 0x48e5, 0xb4, 0xda, 0xdb, 0xec, 0x45, 0xb3, 0xdf, 0x01);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN16,     0xa4fbdbb0, 0xa113, 0x482b, 0xa2, 0x32, 0x63, 0x5c, 0xc0, 0x69, 0x7f, 0x6d);

@Andarwinux
Great! Thanks for the heads up. cc @Nevcairiel

Nevcairiel · 2023-12-14T12:23:03Z

If there is an actual specification, a link to that would be helpful. On a quick glance I couldn't find it, but I didn't have much time to go digging right now.

Andarwinux · 2023-12-14T16:36:59Z

The online specification documentation is not yet available. Microsoft will update the stable Windows SDK shortly, and the documentation will be updated at that time. But for now the new headers needed for Rext support are available from the Canary Windows SDK, and Chromium developers will be writing a new implementation of Chromium D3D12 HEVC Rext next week.

Andarwinux · 2023-12-14T16:47:56Z

Update: DirectX-Headers already contain new redistributable headers.
https://github.com/microsoft/DirectX-Headers/blob/main/include/directx/d3d12video.h#L6304

kasper93 · 2023-12-14T18:17:58Z

Apparently they ship preview SDK on nuget https://www.nuget.org/packages/Microsoft.Windows.SDK.CPP.x64/10.0.25936-preview (includes those new defines), but I feel like there is no point in doing anything until there is proper release and documentation.

Nevcairiel · 2024-03-19T15:31:33Z

At least the latest preview SDK has all the required information - however no drivers exist yet that support it, at least for non-insider windows (and I'm not bothering with that). So it might only become available once the next major update of Windows 11 rolls out.

nyanmisaka · 2024-03-20T18:58:32Z

At least the latest preview SDK has all the required information - however no drivers exist yet that support it, at least for non-insider windows (and I'm not bothering with that). So it might only become available once the next major update of Windows 11 rolls out.

Perhaps we add support by first borrowing the GUID already implemented in Intel's existing driver, and then switch when the new driver & GUIDs becomes available in Windows 11? Driver support for some early Rext-enabled Intel graphics cards such as 10th Gen Ice Lake and Jasper Lake has been moved to maintenance-only status. They may never receive drivers containing these new GUIDs, but their Intel-specific GUIDs are always available.

https://github.com/hughbe/windows-sdk-headers/blob/89f151d343587fcd5b854a4851fcb8f7187f3ac4/Microsoft.Windows.SDK.CPP/latest/Include/um/dxva.h#L1273-L1304

Still, I haven't carefully compared the differences between the old and new DXVA_PicParams_HEVC_RangeExt structures. Correct me if I'm wrong.

Nevcairiel · 2024-03-20T23:38:43Z

Unfortunately the Intel version of that structure is not compatible with the Microsoft version, so I won't be offering support for those.

nyanmisaka · 2024-03-25T06:33:13Z

Unfortunately the Intel version of that structure is not compatible with the Microsoft version, so I won't be offering support for those.

That makes sense. They are just incompatible.

SA-Root · 2024-05-28T14:33:30Z

It seems that HW decoding for HEVC 444 is now working using the D3D11 decoder in Potplayer. I'm using latest public Intel driver version 5522(https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html). So I assume the driver is no longer a problem. Hope you guys can get it working soon.

nyanmisaka · 2024-05-28T14:48:38Z

"MPC Video Renderer" uses Intel-specific HEVC Rext GUIDs.

SA-Root · 2024-05-28T15:04:54Z

"MPC Video Renderer" uses Intel-specific HEVC Rext GUIDs.

Well ,I switched to D3D11 Video Render this time, and it seems to still be using HW decode.

nyanmisaka · 2024-05-28T15:16:39Z

PotPlayer, as a closed source software, also supported Intel-specific GUIDs not long ago.

Nevcairiel · 2024-05-28T15:25:31Z

Intel actually supports a good chunk of the new GUIDs already, but there is still no official spec on their limitations etc, which I expect will arrive with the next Windows 11 major update (24H2), which will be coming out sometime in the second half of this year.

  -> HEVC / H.265 variable-length decoder, main
  -> HEVC / H.265 variable-length decoder, main10
  -> HEVC / H.265 variable-length decoder, main10 extended
  -> HEVC / H.265 variable-length decoder, main10 422
  -> HEVC / H.265 variable-length decoder, main 444
  -> HEVC / H.265 variable-length decoder, main10 444
  -> HEVC / H.265 variable-length decoder, main12
  -> HEVC / H.265 variable-length decoder, main12 422
  -> HEVC / H.265 variable-length decoder, main12 444

mirh · 2024-08-10T23:19:01Z

jellyfin/jellyfin-ffmpeg#202
https://code.videolan.org/videolan/vlc/-/commit/b693265a387f74e39cbe9d8cd37d0c0190ee4a0d
This has been supported since a long time (and there was a recent attempt tried by the MPC-BE guy)

Hooking up that seems more important than whatever W11 (and Xe likely?) only path.

nyanmisaka · 2024-08-10T23:33:13Z

jellyfin/jellyfin-ffmpeg#202 https://code.videolan.org/videolan/vlc/-/commit/b693265a387f74e39cbe9d8cd37d0c0190ee4a0d This has been supported since a long time (and there was a recent attempt tried by the MPC-BE guy)

Hooking up that seems more important than whatever W11 (and Xe likely?) only path.

New generic GUIDs for Rext also appeared on W10 (UHD7xx, Xe and newer), but there is no document on how to use these interfaces. FFmpeg does not want to accept Intel-specific interface definitions. So either you patch FFmpeg yourself, or wait for Microsoft to update the documentation and developers to submit new patches.

Nevcairiel · 2024-08-11T05:22:35Z

Hooking up that seems more important than whatever W11 (and Xe likely?) only path.

An Intel-only proprietary method which is also entirely undocumented and incompatible with the official support seems not more anything.

As I've said before, the Intel-specific version will not be supported. The official specification from Microsoft is almost here, and all current Intel drivers support it on all hardware thats still getting new drivers. My UHD 770 iGPU certainly exposes it. Its also not limited to Windows 11, as all it really is, is a specification of how data is exchanged, it does not require OS support.

In any case, hopefully Microsoft updates the specification document soon to give additional guidance on how to use it, but even if they don't its not entirely impossible to implement otherwise, just will take more time figuring stuff out.

mirh · 2024-08-12T19:57:02Z

FFmpeg does not want to accept Intel-specific interface definitions.

I haven't seen that stated anywhere (and the relative issue is still open and got acknowledged happily by one of the contributors). As with many other ffmpeg work, it just seems like that the authors gave up after the first round because boy isn't there friction for doing everything there.

Though now that I check Intel's docs better.. 422 and 444 only started to be supported with ice lake. And so that would be the only generation left out by their newer drivers if any.

cgutman · 2024-08-13T05:41:50Z

Here is an implemention of the new standard DXVA HEVC Range Extension profiles introduced in the Windows 11 24H2 SDK: cgutman/FFmpeg@9531ce6

Tested with HEVC Main 4:4:4 and HEVC Main 10 4:4:4 content on an Intel Arc A770 and Intel Alder Lake iGPU on driver version 32.0.101.5768.

It needs some more polish before it's suitable for upstream FFmpeg (proper profile detection and DXVA2 support, at least).

nyanmisaka · 2024-08-13T06:58:50Z

Here is an implemention of the new standard DXVA HEVC Range Extension profiles introduced in the Windows 11 24H2 SDK: cgutman/FFmpeg@9531ce6

Tested with HEVC Main 4:4:4 and HEVC Main 10 4:4:4 content on an Intel Arc A770 and Intel Alder Lake iGPU on driver version 32.0.101.5768.

It needs some more polish before it's suitable for upstream FFmpeg (proper profile detection and DXVA2 support, at least).

In order to determine the exact profile, we also need a func like this.
https://github.com/FFmpeg/FFmpeg/blob/82085a3e0a5a2d167dc51628678f7374dc53060e/libavcodec/vaapi_hevc.c#L591-L654
As for dxva2, it is supported and tested in my intel-specific patch.
https://github.com/jellyfin/jellyfin-ffmpeg/blob/jellyfin-7.0/debian/patches/0037-add-d3d11va-dxva2-hevc-422-444-decoding-on-intel.patch

mitzsch mentioned this issue Jul 26, 2023

[D3D11VA] HEVC YUV444 hardware decoding not working mpv-player/mpv#12029

Closed

tophf mentioned this issue Dec 5, 2023

Auto-fallback to NVDEC for HEVC YUV444 to enable GPU decoding? Aleksoid1978/MPC-BE#334

Closed

mirh mentioned this issue Aug 10, 2024

Support YUV 4:4:4 streaming moonlight-stream/moonlight-qt#1282

Merged

6 tasks

Andarwinux mentioned this issue Sep 8, 2024

Missing d3d12video.h? mingw-w64/mingw-w64#24

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Missing 4:2:2/4:4:4 DXVA hwaccel support on Intel graphics #547

Missing 4:2:2/4:4:4 DXVA hwaccel support on Intel graphics #547

nyanmisaka commented Jul 3, 2023

Nevcairiel commented Jul 4, 2023

nyanmisaka commented Jul 4, 2023

nyanmisaka commented Jul 4, 2023

nyanmisaka commented Jul 4, 2023

Andarwinux commented Dec 14, 2023

nyanmisaka commented Dec 14, 2023

Nevcairiel commented Dec 14, 2023

Andarwinux commented Dec 14, 2023

Andarwinux commented Dec 14, 2023

kasper93 commented Dec 14, 2023

Nevcairiel commented Mar 19, 2024

nyanmisaka commented Mar 20, 2024

Nevcairiel commented Mar 20, 2024

nyanmisaka commented Mar 25, 2024

SA-Root commented May 28, 2024

nyanmisaka commented May 28, 2024

SA-Root commented May 28, 2024

nyanmisaka commented May 28, 2024

Nevcairiel commented May 28, 2024

mirh commented Aug 10, 2024

nyanmisaka commented Aug 10, 2024

Nevcairiel commented Aug 11, 2024

mirh commented Aug 12, 2024

cgutman commented Aug 13, 2024

nyanmisaka commented Aug 13, 2024

Missing 4:2:2/4:4:4 DXVA hwaccel support on Intel graphics #547

Missing 4:2:2/4:4:4 DXVA hwaccel support on Intel graphics #547

Comments

nyanmisaka commented Jul 3, 2023

Nevcairiel commented Jul 4, 2023

nyanmisaka commented Jul 4, 2023

nyanmisaka commented Jul 4, 2023

nyanmisaka commented Jul 4, 2023

Andarwinux commented Dec 14, 2023

nyanmisaka commented Dec 14, 2023

Nevcairiel commented Dec 14, 2023

Andarwinux commented Dec 14, 2023

Andarwinux commented Dec 14, 2023

kasper93 commented Dec 14, 2023

Nevcairiel commented Mar 19, 2024

nyanmisaka commented Mar 20, 2024

Nevcairiel commented Mar 20, 2024

nyanmisaka commented Mar 25, 2024

SA-Root commented May 28, 2024

nyanmisaka commented May 28, 2024

SA-Root commented May 28, 2024

nyanmisaka commented May 28, 2024

Nevcairiel commented May 28, 2024

mirh commented Aug 10, 2024

nyanmisaka commented Aug 10, 2024

Nevcairiel commented Aug 11, 2024

mirh commented Aug 12, 2024

cgutman commented Aug 13, 2024

nyanmisaka commented Aug 13, 2024