WebGPU renderer support

[danrossi]

There is a new renderer for webgpu support which I've seen uses less resources. But on integration as a custom renderer with troika I've seen 30-70% usage. I'm not sure what part causes that problem.

I've built a bundle for the new renderer to try.

https://github.com/danrossi/three-webgpu-renderer

I set it up within a World3DFacade with

this.rendererClass = THREE.WebGPURenderer;
this._threeRenderer = new THREE.WebGPURenderer({ canvas: canvas, alpha: true,
	        antialias: true });

I've created a project demonstrating the webgpu renderer but the GSL for the text blocks don't work with webgpu. Image and video textures are ok just not text.

https://github.com/danrossi/three-troika

[danrossi]

I've included a webgpu renderer example. The text shader is not rendering properly yet and the text and background is showing us a black and white squares. A video texture performance issue has been fixed for cpu but uses alot of gpu. That should hopefully be fixed soon also. https://github.com/danrossi/three-troika/blob/main/test/webgpu.html

[danrossi]

Example webgpu with video texture. The text rendering has an issue. https://danrossi.github.io/three-troika/test/webgpu.html

[danrossi]

I've got a better example of the Webgpu integration working. But text is showing up as white and black squares. Isolated example with the text package. What could be causing this ?

https://danrossi.github.io/three-troika/test/webgpu_text.html

[lojjic]

Are existing custom GLSL shaders expected to work automatically with WebGPU?

[danrossi]

It's a transformed MeshBasicMaterial. I might have to ask the question there first. I'm pretty sure its converting what is in meshbasicmaterial. Or it requires to be done in nodes which is complicated. I have another sdf method I use that used RawShaderMaterial and will see if I can get that working first.

[danrossi]

onBeforeCompile is not run for webgpu renderer. I'll do more digging.

[danrossi]

It's extensive but the material needs to become a MeshBasicNodeMaterial. I believe its then portable to both renderers. I'll look into it for the simpler bmfont shader first.

mrdoob/three.js#22603

[danrossi]

After alot of mucking around I figured out the solution for building a custom shader for WebGPURenderer. To support both WebGPU and WebGL backends it can be done in a universal tslFn method with imports to handle uniforms and common math methods. custom shader methods I haven't figured out so all inline is generated. I've implemented it into my bmfont fork for now. There is slight differences to uv geometry between both backend I found.

So. until the api is more polished and supports WebXR WebGLRenderer is still needed with custom shader. Then it can just implement the nodes system to dynamically generate the shader instead.

https://danrossi.github.io/three-webgpu-renderer/tests/webgpu_text.html
https://danrossi.github.io/three-webgpu-renderer/tests/webgpu_gl_text.html

https://github.com/danrossi/three-bmfont-text/blob/502c5697e3c52516fccd986129e85a4a5f41f61f/src/shaders/MSDFShader.js#L69
https://github.com/danrossi/three-bmfont-text/blob/502c5697e3c52516fccd986129e85a4a5f41f61f/src/TextBitmap.js#L66

[danrossi]

I had a look how much stuff is packed in those shader hacks and there is quite a bit to port over. Everything works in WebGPU apart from text. I might have to implement my simpler bmfont fork changes into a custom Facade for now if using webgpu.

[danrossi]

I tried to integrate mine but can't get it scaling and displaying right and the background color layer is a custom shader. So have to try and upgrade the whole text thing to webgpu converting what is there to the nodes system.

[donmccurdy]

Are existing custom GLSL shaders expected to work automatically with WebGPU?

@lojjic No, three.js cannot transpile GLSL to WGSL. In THREE.WebGPURenderer, three.js supports a higher-level shading language (three.js shading language or "TSL") which it can compile to both GLSL and WGLSL. TSL is supported in THREE.WebGPURenderer, which has fallback rendering WebGL 2, and transpiles from TSL to GLSL300 in that case. Or shaders can be written directly in WGSL, but there's no fallback to GLSL.

For now, most three.js users are still using THREE.WebGLRenderer. There's also support for loading MaterialX shader graphs, but this is limited to THREE.WebGPURenderer (with fallback to WebGL2) as well.

Do you have a preference on how (or if?) WebGPU support should/could be added Troika? Some paths forward could include:

1. Add new shaders/materials for WebGPURenderer, written in [TSL | WGLS | MaterialX]
2. Duplicate shaders is too much maintenance, so no support for WebGPU at least until WebGPURenderer becomes the default three.js renderer

I personally prefer (1), but would totally understand either choice. Some idea of what you'd prefer would be very helpful. Thanks!

[danrossi]

I tried and failed to get the text shader working with webgpu. Very convoluted system. Took ages. It has to be turned into a node system to build the shader. I think there is some other modules that use custom shaders also. Basically RawShaderMaterial doesn't work with webgpurenderer. I made a few attempts here trying to convert the shader to a nodes system

https://github.com/danrossi/three-webgpu-renderer/blob/master/tests/webgpu_nodes.html#L169

[powersimple]

I've been following Troika for years. I'm directly connected to the Google team working on WebGPU. If if someone can formulate this thread into asks I can bring it to them. We can't have WebGPU soon enough.
Also, WebGPU doesn't support VR yet. If anyone here is attempting that, this in particular, I'm advocating happen as soon as feasible.

[donmccurdy]

GLSL shaders must be rewritten for WebGPU, using GLSL directly in WebGPU is not possible and probably will never be. Examples of MSDF-based font rendering for WebGPU can be found here:

https://webgpu.github.io/webgpu-samples/?sample=textRenderingMsdf

Whether the cost of maintaining additional code for WebGPU is worthwhile, I think that's up to the maintainers to decide.