GPU: Enable use of shared resources

[JBetz]

Feature request: Enable SDL_gpu to use textures and other resources created in other processes on the same GPU.

Metal: https://developer.apple.com/documentation/iosurface

Direct3D12: https://learn.microsoft.com/en-us/windows/win32/api/d3d12/nf-d3d12-id3d12device-opensharedhandle

Vulkan: https://registry.khronos.org/vulkan/specs/latest/man/html/VK_KHR_external_memory.html

Use case: CEF provides an accelerated paint function that passes a handle to a shared texture with frame data for a rendered webpage. This is an alternative to the normal approach, which is copying a texture from CPU and then back into the GPU on every frame, which is obviously inefficient and often results in stuttering when streaming video.

However, there's currently no way to create a SDL_GPUTexture from the handle with SDL_gpu, so the only option is write a D3D11 or D3D12 renderer to use instead to benefit from this feature.

[slouken]

grin @thatcosmonaut

FWIW, this is one of the reasons the GPU renderer hasn’t replaced the D3D/Metal/Vulkan 2D renderers

[JBetz]

@slouken Interesting, I didn't know that this was already possible with the 2D renderer! I'll give it a try to compare performance with my SDL_gpu renderer. Of course, a SDL_gpu solution would be better but at least I can prove the value of it for my use case.

WRT to CEF integrations specifically, is it possible to copy the shared texture, or do you have to render it immediately? I've read that in recent Chromium versions the texture is no longer reused and you have to assume a new texture on every frame. But looking at the 2D renderer API, I don't see any way to copy it a la SDL_gpu's SDL_CopyGPUTextureToTexture.

[slouken]

With the 2D renderer you create a new SDL texture for each external texture handle. It directly wraps the native handle and contains the contents of that texture, although there is no explicit synchronization, so if you need to wait on a shared mutex you should do that outside of SDL code.

[JBetz]

@slouken I'm trying to use a shared texture with SDL_Renderer but am getting an invalid memory access error if I just pass the texture handle from CEF directly to SDL via the SDL_PROP_TEXTURE_CREATE_D3D11_TEXTURE_POINTER property.

Presumably, I need to call ID3D11Device::OpenSharedResource first, but in order to do that, I need a reference that D3D11Device that SDL is using, which AFAICT is not exposed.

Am I doing something wrong, or is this workflow just not possible with the current API?

[slouken]

Presumably, I need to call ID3D11Device::OpenSharedResource first, but in order to do that, I need a reference that D3D11Device that SDL is using, which AFAICT is not exposed.

Take a look at SDL_PROP_RENDERER_D3D11_DEVICE_POINTER

[JBetz]

@slouken Thanks! That unblocked me. Now I think it's just a matter of figuring out my platform's C++ FFI to make sure I'm calling OpenSharedResource correctly.

I have a newfound appreciation for SDL_gpu having discovered that Direct3D is a C++ API. Virtual tables are no fun.

UPDATE: Got it working. Ended up writing a C wrapper for all of the Direct3D11 interfaces rather than interfacing with C++ directly. See https://magpcss.org/ceforum/viewtopic.php?f=6&t=19404&start=40#p57468 for the source.