[Feature Request] SDL3 GPU Backend for WebGPU Target

[ShadowMarker789]

Issue created to track SDL3 GPU Backend, targeting WebGPU.

Extract from SDL_GPU Readme page

Future plans are to also support:
• WebGPU

SDL3 GPU is a very exciting project, and it would be great to add WebGPU as a compilation target.

Many thanks to the SDL Team for their hard work and great accomplishments.

[icculus]

This is definitely something I want, but I reasonably doubt anyone will have time to work on this before SDL3 ships.

I'm going to assign this to myself and put it in the 3.2.0 milestone, though I believe both actions are delusional on my part.

[slouken]

FYI: https://x.com/kylelukaszek/status/1832979451203686676

[ericoporto]

Yeah, apparently development is happening here

https://github.com/klukaszek/SDL
https://github.com/klukaszek/SDL3-WebGPU-Examples

@klukaszek

[leandro-benedet-garcia]

FYI: https://x.com/kylelukaszek/status/1832979451203686676

Sorry, could you post a screen shot? Twitter is banned in my country.

[ShadowMarker789]

FYI: https://x.com/kylelukaszek/status/1832979451203686676

Sorry, could you post a screen shot? Twitter is banned in my country.

Video to the right shows a rapid flickering of various colors, like one is randomly picking a color and clearing the screen with it.

[klukaszek]

Hey! I’ve been working on this in my spare time and I’ve managed to get the driver entry working for the example test suite. I’m planning on getting to work on pipeline creation tonight after I do my homework lol.

[klukaszek]

Cycling between demos using A and D will result in failure due to missing function implementations for the GPU driver. I also had to kind of hack in a way to prevent mouse events from flooding the event queue, but I believe this has to do with the browser and not SDL itself.

[klukaszek]

The crashing mentioned in the earlier tweet was a result of a problem where any event would trigger a resize event causing swapchain recreation. This has been solved.

So far I’ve tested resize, mouse, and key events and they all work as if using SDL2 on web.

Sorry for the spam, I’m just writing from my phone waiting for class.

[icculus]

Talk all you want, this is awesome. :)

[flibitijibibo]

Something that may help move this along is migrating our examples repo over:

TheSpydog/SDL_gpu_examples#12

Since we presumably need to make changes for Emscripten and the examples folder has automation for exporting web samples, doing these together seems like a good idea.

[klukaszek]

Here's the latest update on my WebGPU progress.

https://x.com/KyleLukaszek/status/1839631105646862343

Video for those of you who don't have access to Twitter:

2024-09-27.07-18-37.mp4

Update

I ended up DM'ing with @ beaufortfrancois (Google Chrome) and he told me that the SPIR-V support for Chrome was bugged and never worked properly. The team decided to scrap SPIR-V support altogether and will be removing it from Chromium as of the next release.

This presented an interesting problem. SDL uses SPIR-V as its shader type, but browsers will require a valid WGSL WGPUShaderModule to process the shader.

I ended up porting Google's Tint WGSL compiler with limited functionality to WASM and linking it with the Emscripten build of SDL. This allowed me to access the Tint C++ API and export wrapper functions to C for converting SPIR-V bytecode to WGSL.

This is not an elegant solution, but I did not feel like completely changing how shaders are handled in SDL.

Anyway, I am mostly concerned at this point about being 100s of commits behind the main branch now. Any tips for syncing?

[klukaszek]

Here’s a live demo to test the current status of the example suite.

www.kylelukaszek.xyz

[flibitijibibo]

For the WebGPU backend you'll likely end up adding a WGSL value to the bitflags, so shaders can be passed directly to SDL without needing an extra compiler (that can be the app's problem).

[Akaricchi]

This is not an elegant solution, but I did not feel like completely changing how shaders are handled in SDL.

SDL's shader policy is to expose the backend's native shader format. The right way to do it is to add a new SDL_GPUShaderFormat constant for SDL_GPU_SHADERFORMAT_WGSL and have users provide WGSL shaders. Converting SPIR-V into other backend-specific shader formats at runtime is a responsibility of SDL_gpu_shadercross, it shouldn't be done by SDL itself. The clients can choose to use that, or generate their shaders offline with whatever workflow suits them.

Anyway, I am mostly concerned at this point about being 100s of commits behind the main branch now. Any tips for syncing?

Use git rebase. I advice enabling git rerere before doing that to automate future conflict resolution. I'd also recommend squashing your small fixup commits and organizing them roughly by added functionality.

[klukaszek]

Yeah I’m gonna end up moving the shader cross-compilation to SDL_gpu_shadercross now that I know that the Tint compiled shaders run properly.

As for loading WGSL shaders, I’ll just add a new enum constant to SDL_GPUShaderFormat and pass the WGSL code as a Uint8* and convert it back to a string later. I’ll work on getting that working this weekend as well as rebasing.

Thanks for the help!

[klukaszek]

I managed to rebase and get everything working again with the latest commits!

Next, I'll work on removing the shader compilation from SDL and moving it to SDL_ShaderCross.

I have a few concerns about how Tint should be integrated in SDL_ShaderCross, but I'll stick to linking with a static library of a minimal Tint build + custom C bindings for compiling SPIRV to WGSL.

[klukaszek]

I moved all shader compilation logic over to ShaderCross and updated the WebGPU backend to have WGSL as the expected shader format.

I have tested loading WGSL from a file directly into SDL, as well as loading SPIRV from a file. SPIRV is passed through the ShaderCross layer, and then converted to WGSL. This is because ShaderCross recognizes the backend shader format as WGSL and converts accordingly.

This should follow SDL's shader policy much better now.

The current web demo is still converting from SPV to WGSL.

[slouken]

Is your work ready to be a PR and potentially merged? It would be nice to ship with WebGPU support out of the box in the upcoming preview release.

[klukaszek]

The demo is currently only working for the most basic of shaders so I’m not sure if it’s entirely ready yet.

My SDL fork should be good for a PR though if you want to add very rudimentary support for the time being. Might be a few commits behind but that’s not a big issue.

[klukaszek]

As for ShaderCross,

My current solution for loading Tint into ShaderCross uses a static library as opposed to using SDL_LoadObject() since WASM requires the use of static libraries, not dynamic ones.

Since we only worry about WGSL for browser builds (for the time being at the very least) I can serve a static WASM library from GitHub to avoid the nightmare of having to build Tint for WASM from source.

This is all handled by a Makefile that clones the lib repo, compiles our C wrapper functions to a .o file, adds the .o file to the existing Tint library, copies the lib to SDL_gpu_shadercross/, and finally deletes the cloned repo.

If you’re building for native, this should all be ignored.

The ShaderCross WGSL functionality currently only supports SPIR-V and WGSL. If you want to try and add other Tint backends, please do, but the static library size will be massive (and might actually not work whatsoever).

These changes to ShaderCross could most likely be PR’d to the appropriate repo if the inclusions seem acceptable.

[thatcosmonaut]

Is your work ready to be a PR and potentially merged? It would be nice to ship with WebGPU support out of the box in the upcoming preview release.

I don't think there should be any rush to get this in, this is going to be a lot of work and I don't want to cram in WebGPU at the 11th hour and have clients expect it to be ready to use.

[icculus]

I agree, take the time to do it right.

[TheSpydog]

These changes to ShaderCross could most likely be PR’d to the appropriate repo if the inclusions seem acceptable.

Sure, we can take a look.

Out of curiosity what kind of binary sizes are we looking at for a wasm-ified Tint? From my experience it's very heavyweight on Windows, but granted that's with more than just the spirv/wgsl systems enabled.

[klukaszek]

Out of curiosity what kind of binary sizes are we looking at for a wasm-ified Tint? From my experience it's very heavyweight on Windows, but granted that's with more than just the spirv/wgsl systems enabled.

GitHub says that the libtint.a static library is 37.5 Mb.

When compiled with the test suite, the resulting WASM binary that is produced is 12.5 Mb.

When this WASM file is served by GitHub Pages, Chrome network tools reports that the final WASM binary is 2.7 Mb (it's still 12.5 Mb in reality though).

This includes SDL, ShaderCross + Tint, and the example suite (+ the static content).

[klukaszek]

Update

Vertex buffers are now operational! This means that examples 3 & 4 now work on the demo site.

Demo 4 is restricted to a 320x480 window so do not worry if the drawn output does not fit the canvas.

Haven't checked out what I have to do next but I've got two math midterms to study for so I might disappear for a bit. We'll see though.

Rambling

I got stuck trying to get buffers working for quite some time, even though I could have sworn that my implementation was fairly sound... As it turns out, there is a WebGPU regression in Emscripten versions >= 3.1.65 responsible for my pain. I opened an issue in the Emscripten repo so hopefully it will get resolved soon.

Turns out there were more than just 1 WebGPU regression in the last few updates to Emscripten so for the time being I'll just stick to 3.1.64 and occasionally test the latest release.

[ryuukk]

Emscripten is not WASM, it is wrong to have it as a dependency, or to assume that's what everyone will use

Take a look at sqlite for how to properly target WASM as a library author:

https://sqlite.org/wasm/doc/trunk/building.md

You should provide the javascript glue code manually, you don't just shove emescripten and call it a day, it is platform and language specific, not portable

More documentation on WASM without emscripten:

https://schellcode.github.io/webassembly-without-emscripten

[klukaszek]

You should provide the javascript glue code manually, you don't just shove emescripten and call it a day, it is platform and language specific, not portable

Respectfully, I don't think reimplementing existing JS bindings for WebGPU Headers would be a good use of my time (or anyone's).

I was planning on going through and removing most references to Emscripten-specific function calls, but if I was using it for the WebGPU bindings anyway, then there was no point in doing so.

Emscripten is not WASM, it is wrong to have it as a dependency, or to assume that's what everyone will use

The SDL repository already references Emscripten (CMakeLists.txt), so basing the WebGPU backend off of it is not my biggest concern at the moment.

SDL_gpu_webgpu.c references a total of 3 Emscripten-specific functions and 1 struct. 2 of those function calls are emscripten_sleep() which can be swapped out with SDL_Delay(). The other is wgpuDeviceCreateSwapChain(), which is an Emscripten abstraction used to create a WGPUSurface to use as the swapchain. So once everything seems stable, I can start picking away at the Emscripten-specific stuff.

As for Tint-WASM, I can look into that another time since (again) I'm already using Emscripten to compile SDL, so using it to compile the examples just works. I do agree that Tint-WASM should probably be tested with other WASM compilers to see if the output works as intended.

If you could point me to any other WebGPU JS bindings for WASM I would greatly appreciate it. I'd be more than happy to check them out, those links you provided seem very informative!

[klukaszek]

I finally managed to get textures to appear but there seem to be some issues that need to be ironed out.

BindGroups are very rudimentary at the moment and will be worked on over time to support more and more binding types. So far only textures, buffers, and samplers are supported. Storage variants are not yet supported.

As always, you can use an extension like WebGPU Inspector and check out the demo site to see what's happening under the hood. Some resources aren't cleaning up properly at the moment so I'll also have to go in and fix that.

[klukaszek]

I had some time recently to sit down and fix the texture issue. I can continue some more after I finish preparing for my term tests.

Ravioli in the browser!

[slouken]

Good luck with your term tests!

[eliemichel]

Oh I wasn't aware of this thread at the time but I did recently a sdl3webgpu minilib (just a .h and .c) to create a WebGPU surface from a SDL3 window. There is also a similar one for SDL2: sdl2webgpu

[klukaszek]

@eliemichel That's great since my current implementation only supports Emscripten! I have not touched WebGPU Native with SDL3 yet, but sdl3webgpu.c seems to be more than enough to cover that. I can rewrite the WGPUSurface section of my code to use the logic of your sdl3webgpu.c to handle the surface creation. This should eliminate the dependencies on Emscripten-specific WebGPU functions and introduce interoperability with WebGPU Native & Dawn.

On another note, for everyone else, I had the chance to sit down and fix the samplers in the texture demo.

2024-11-25.21-32-11.mov

I hope to find more time to progress on this before exams.

[onehundredfeet]

any idea when this will be in a usable state? I'm very interested. Specifically the Emscripten version

[klukaszek]

@onehundredfeet My goal is to at least have vertex and fragment uniform buffers in a solid state before the next semester starts. It should be my last semester thankfully, but I will be TA’ing so I’m not sure how much time I’ll have to work on this.

Overall, I would estimate that I’ve implemented 50-60% of the SDL_GPU API, but some of the current implementations may have to be revised. At the very least, I’d expect a few more months until it’s “usable” for most tasks.

I should also clarify that I’ve switched to converting the HLSL shaders by hand since the WGSL cross-compiler solutions rarely work smoothly. Once I incorporate native support, we should be able to provide SPIR-V bytecode with no issues as we won’t be bottlenecked by the browser backends.

[onehundredfeet]

Thanks for the info. I use slang to transpile into target languages. Have you seen it?

If you published a simple roadmap in the readme I may be able to contribute a bit if I knew which areas needed work but you haven't gotten to yet.

[klukaszek]

@onehundredfeet Yeah I saw that Slang released their WGSL transpiler! I was thinking of testing it out when it came out, but I was working on some other things at the time… It’s not too big of an issue, though I will check out Slang sometime soon.

As for the roadmap, I’ll beautify my checklist from my notes and add it to the readme file!

Edit: I rebased with SDL's upstream main branch, and added a checklist to the WebGPU fork. This latest branch has all of my unpushed local changes. Currently, I am still fighting with uniform buffer management with bind groups.

[klukaszek]

I finally managed to implement SDL_GPUBlit() for 3D textures! WebGPU doesn't have any in-built blit functionality like Vulkan, so it was a pain to figure out how I should approach it until I realized I could just write a graphics pipeline to solve the issue. Next, I've gotta add some pipelines for 2d, 2d_array, cube, and cube_array textures, and write some tests if none already exist for those.

Edit: They do exist :)

The current overhead from the custom blit method is the creation of a bindgroup, a command encoder, and a command buffer. The pipelines, samplers, and uniform buffer should only be created once and exist within some blitCache struct. Might have to add some hashing to figure out when the cache is invalid and the pipelines need to be rebuilt.

Demo Site (Open the console!)

[TheSpydog]

If you hadn't seen it, there's an SDL_BlitCommon internal function that D3D12/Metal use, which automatically calls the SDL-level graphics pipeline creation / uniform / draw functions for you. You just need to set up your shaders in the same way as Metal_Blit and D3D12_Blit. There's a fair amount of subtlety involved so I'd recommend using that instead of rolling your own blit implementation, if possible!

[klukaszek]

Ah I see that now, and it is essentially implemented just as I was planning on doing to extend my blitCache! I'll try to refactor everything over to SDL_GPU_BlitCommon and see how it works out. I guess I've gotta put together the shaders for all pipelines first.

[klukaszek]

I managed to get BlitCommon working with my backend which is a huge relief. Now I just need to do some bug squashing to fix errors like the following:

[klukaszek]

Update:

Current:

Expected:

[klukaszek]

BlitCube works!

2025-01-10.05-25-01.mp4

[flibitijibibo]

This is looking good! FWIW I think we're at a stage where this can be a draft PR no matter how messy it is; there are probably things we can upstream early and with a fresh GitHub ticket you'll have a nice place to throw together a roadmap for others to see and maybe help out with.

[klukaszek]

Awesome! I'm waiting for the git functionality outage to be resolved so I can push my latest commits. Once that's done, the only failures in the pipeline should be FreeBSD (I don't know why it's yelling at me) and Emscripten (I think it's a versioning issue).

Edit: I confirmed that these are the only pipelines that fail.

[klukaszek]

I added a bit more functionality to the backend (indirect drawing), and I've now opened a PR with my work!

Emscripten swapchain-specific function calls are no longer used either. We now create a surface and configure it properly instead of calling wgpuSwapChainXXX() functions provided by Emscripten.