[3.x] Shader goodies: async. compilation + caching (ubershader approach)

[RandomShaper]

This PR, by being a better implementation of #46330, supersedes it. All the feedback from maintainers that was sent there has been addressed in this one. The approach to a reasonable fallback during background compilation that this new implementation features is ubershaders (details below), as suggested by @reduz.

---

The main goal of this PR is to reduce stalling in games.

UPDATE: Screenshots updated to reflect latest changes.

Shader caching

As long as the target platform supports the program binary GL extension, this is just enable and forget.

Some remarks:

• Writes to the cache are async. to prevent stalling the render thread as much as possible. UPDATE: The cache is now only used for ubershaders (see below).
• If async. shader compilation is enabled in addition to caching, shader "reconstruction" from its cache file is also potentially done asynchronously.

Asynchronous compilation of shaders

It will work if enabled and supported by the GL driver. If native parallel compilation is supported, that's used, which is the most efficient. Otherwise, asynchronicity is achieved via a secondary GL context (and another thread) that sends the compiled shader back to the main one in its binary form, which means the program binary extension must be supported. If both fail, async. compilation is effectively disabled.

Three fallback modes are added to both manually created shaders (either codey or visual) and SpatialMaterials: ubershader, none and no render. UPDATE: Two fallback modes are added to both manually created shaders (either codey or visual) and SpatialMaterials: visible (ubershader), hidden.
Please check the diff where these are explained in the built-in documentation.

The default mode is visible, which is the main highlight of this PR. It hints the engine to use a universal version of the base shader (e.g., scene) that supports all the possible contexts (rendering conditionals), so it's slower than the properly conditioned version of the real shader, but ready from the very beginning to be used while the real deal is being compiled in the background.

Some simple parsing of special directives has been added to the GLSL language so the base shaders can be transformed on the go from a preprocessor based handling of conditionals to a runtime one.

You can explicitly set a different conservative mode for any shader/material.

UPDATE: Screenshots updated to reflect latest changes.

Please also see the diff for an explanation of the different project settings.

Testing

Since both caching and async. compilation are experimental at this point, they are disabled by default. Please test this in your own projects with and without caching. A good starting point would be to enable caching for the editor only and keep the default number of simultaneous compiles. Depending on the target hardware, among other things, those may be raised to find the perfect balance for any project.

Future work

These would be good for future PRs:

• Ubershaderify other shaders than the scene one (particles (DONE), canvas).
• Split vertex and shader compilation into discrete states, for finer grained control of CPU usage. (DONE)
• Collect/provide metrics about fallbacks used, compiles in flight, cache retrievals, etc.
• Allow the ubershader to be customized to control the balance between quality and performance.

[QbieShay]

Fallback none and no_render are a bit confusing, perhaps we can find some alternative terms?

[RandomShaper]

Fallback none and no_render are a bit confusing, perhaps we can find some alternative terms?

Good point. Taking inspiration from @Calinou's suggested changes, I'm thinking that a great rename could happen, like this:

• Fallback mode -> Async mode
• Fallback no render -> Async mode invisible
• Fallback ubershader -> Asyc mode visible slow
• Fallback none -> Async mode disabled

Something along those lines. Thoughts?

[QbieShay]

@RandomShaper I'm not very convinced. What about:

Shader Compilation

• Hide until ready (fast)
• Simplified fallback (intermediate)
• Synchronous (slow)

Something like this?

[TokisanGames]

Adding async isn't more clear. And names like slow, fast, etc aren't descriptive. I want to know what will visually appear on screen.

Fallback is clear. This will be used while the full shader is being compiled.

Don't render / invisible is clear.

The other states aren't descriptive.

Rather than offer none/no shader, why not default to a simplified albedo shader and give me a color picker, and call it color.

What is Uber shader? It doesn't mean anything to me. Custom means something.

I would call it a fallback shader, with choice of: "don't render, color, custom"

[QbieShay]

@tinmanjuggernaut That was the previous approach, and it adds more complexity and doesn't work as well as an ubershader for most cases.

[RandomShaper]

Shader Compilation

• Hide until ready (fast)
• Simplified fallback (intermediate)
• Synchronous (slow)

Those look like more intuitive. However, Simplified fallback (intermediate) doesn't quite fit. The fallback is slower, but no simpler. Also, it'd be great that the first two are somehow understood as asynchronous modes. Slow until ready would meet, but sounds a bit strange, doesn't it?

[RandomShaper]

Pushed with documentation that was missing. Naming will be changed when we find good names.

[QbieShay]

Okay, how about

Shader Compilation

• Hide until ready (fast)
• Use General purpose (intermediate)
• Blocking (slow)

[clayjohn]

Regarding the naming discussion, is there even a good reason to expose fallback modes? To me it makes the most sense to have a single flag "hide until shader compiled".

I can't see a reason why anyone would want the entire game to freeze up to wait for a shader to compile over using the Ubershader.

[RandomShaper]

At some point I was asking myself what uses cases would the forced sync support. I couldn't think of any, but I assumed flexibility was better, just in case. By bringing that up you have made me finally believe that it's better just to get rid of it. If one day some user comes across a legit use case, we can always reintroduce it.

If we go that route, would we really want to switch from an enumerated type to a boolean, instead of keeping the enum with two values? The latter, in case of a future addition of new fallback modes, would avoid breaking compatibility. I don't have a strong opinion on this.

[TokisanGames]

Oh, the fall backs are blocking? Invisible is ok, but it might reveal hidden elements in a level design. A non-blocking or already compiled universal shader with just opaque colors is better. A complex, blocking fallback shader is unnecessary. If I wanted a blocking shader, I'd just use the stock renderer.

In addition to async compilation, it would be ideal to have a mechanism to force shader compilation before scene load or on game startup, and a signal on complete. I'm doing this with hacky work arounds described in the other PR, but it would be better if built into the engine.

[clayjohn]

If we go that route, would we really want to switch from an enumerated type to a boolean, instead of keeping the enum with two values? The latter, in case of a future addition of new fallback modes, would avoid breaking compatibility. I don't have a strong opinion on this.

This sounds like speculative future-proofing to me. But given tinmanjuggernaut's comments above, is probably a good idea.

Oh, the fall backs are blocking? Invisible is ok, but it might reveal hidden elements in a level design. A non-blocking or already compiled universal shader with just opaque colors is better.

Only "Fallback none" is blocking, the Ubershader approach loads instantly, looks the same as regular materials, but has worse per-frame performance.

I like the idea of the opaque color fallback (like in RandomShaper's last PR), but I have a feeling it will be unnecessary. In my opinion, we should merge this PR with just the Ubershader and "no render" options and then see if there is a need for something in the middle.

A complex, blocking fallback shader is unnecessary. If I wanted a blocking shader, I'd just use the stock renderer.

I agree, but oof :P

In addition to async compilation, it would be ideal to have a mechanism to force shader compilation before scene load or on game startup, and a signal on complete. I'm doing this with hacky work arounds described in the other PR, but it would be better if built into the engine.

We have discussed precompiling shaders many times and the consensus is that it would just be too hard given how Godot is designed.

[TokisanGames]

We have discussed precompiling shaders many times and the consensus is that it would just be too hard given how Godot is designed.

Which was discussed?

1. Precompiling at dev time and storage? sure.
2. Precompiling at game start?
3. Precompiling on scene load before _ready?
4. Precompiling on demand?

@tcoxon and I are already doing 3, 4 in gdscript. I traverse the tree, get every material, display it on screen, plus move the camera to a birds eye view, and a 360 rotation to attempt to force compile every one, then send a signal. Some of this could definitely be done directly in the engine more cleanly such as identifying every loaded material, and a direct initiation of shader compilation.

Changes done.

[akien-mga]

Thanks! 🎉

[TokisanGames]

As of fee83d0, it looks like this patch is still not enabled in the editor and has no option to enable it, so it lags terribly when opening large scenes.

[RandomShaper]

Enabling this in the editor involves some design decisions (separate settings for editor and game?, using ubershader is OK since in summer cases it's not 100% equal to the real shader?, maybe it's better to just enable a cache of regular shaders with no async support, or allow async with no ubershader but some placeholder shader?, etc.).

So, while I agree it may be a good thing, it's not trivial. You may want to open a proposal with your choices so a discussion can happen to see what the most wanted behavior is.

[TokisanGames]

So, while I agree it may be a good thing, it's not trivial. You may want to open a proposal with your choices so a discussion can happen to see what the most wanted behavior is.

I have started the proposal to discuss it here:
godotengine/godot-proposals#3594