3D Rendering architecture for Godot 4.0 GLES3

[reduz]

Describe the project you are working on

Godot

Describe the problem or limitation you are having in your project

Mobile rendering is difficult and it limits the performance that can be obtained. Likewise, we have Godot users that run it in very old desktop hardware that can't do proper latency hiding.

To make it clearer, the following are common problems for mobile rendering:

• Textures can't switch between sRGB and Linear modes, unlike on desktop.
• Reading from textures in shaders has higher latency due to slow memory and limited or no latency hiding. Only prefetch (passing interpolator to UV) is efficient. This affects reading both textures, reflection probes and shadow mapping.
• High bit depth framebuffers are expensive on mobile and old hardware

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Currently in Godot 3.x the GLES3 renderer uses techniques meant for high end, which makes it slow on mobile. Users need to use GLES2 for obtaining higher performance there.

The goal is to make the new GLES3 renderer for Godot 4.0 high performance on mobile by default, given all mobile devices support this level of API nowadays.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

The 3D rendering process in Godot 4.0 GLES3 should be as follows.

Render Passes

The base pass will be able to render:

• Environment
• Lightmaps
• Reflections (up to two reflection probes)
• One directional Light (with shadow).
• Multiple directional lights without shadow
• One positional light (with shadow)
• Multiple positional lights without shadow

Extra lights (directional or positional) with shadow mapping will be rendered in separate passes. There are two main reasons for this:

• Shadow mapping required pre-fetch (reading from an interpolator)
• Because using a texture atlas and dual paraboloid shadow mapping may result in troubles with the prefetch, lights will contain their own shadowmaps in GLES3, and shadows for positional lights will be rendered to cubemaps (no texture atlases used).

The extra plus on all this is the fact that large part of the shadow computation can be done in the vertex shader and interpolated towards the fragment one via varying (GLES3 specification supports by default a larger number of varyings).

Base ubershaders (With all enabled, togglable with bool) will be:

1. Base Pass No Light + Directional + Omni + Spot + Refprobes + Lightmap + Environment
2. Directional + Omni + Spot Additive
3. Shadow (Depth) Pass

Then specialized versions with other features turned off (lightmap, fog, refprobes, etc, specific light combinations) will be handled by the background compiler.

Tone Mapping

We have some restrictions that make tone mapping very difficult:

• HDR using 32bits framebuffers is always extremely limited (RGBA8 and RGB10A2).
• Mobile GLES3 does not support reading a texture in both SRGB and Linear, and for simplicity Godot 4 assumes this is possible.
• Tone mapping is limted anyway with such low bit depth and given Godot renders 3D and 2D separate, using the Godot 3 GLES3 approach of first rendering 3D, then copying (tonemapping) and rendering 2D over it is inefficient.

As such, the following workflow is proposed instead:

• Textures always imported as sRGB, no need to discern between Linear and sRGB
• Before light equations take place, light is converted to Linear using an approximation, then lighting happens, then tonemapping happens and finally conversion back to sRGB happens before writing to the color buffer.
• The main drawback of this approach is that the addition of more lights will be relatively incorrect, but its an acceptable tradeoff.
• If Glow is used, it will be done in sRGB space, like in Godot 3 GLES 2.0.
• Auto exposure will not be supported in GLES3.

This mechanism allows sharing a single RGBA8 (or RGB10A2 depending on configuration) color buffer for 3D and 2D and achieve the best possible performance on old hardware.

Skeletons, Blend Shapes, Particles

Currently in GLES3, skeletons are applied in the vertex shader. This has some problems:

• If you do shadow mapping, skeleton must be processed per light on complex models and this is slow
• vertex texture fetch is relatively slow
• with multipass rendering, it gets even slower
• Supporting Blend Shapes is difficult
• Objects that do not move will still need use this.

So, the idea is to do skeletons using transform feedback in a similar vein to how Vulkan based back-end does it with compute. This makes it easier also to support Blend Shapes and 8 weights per vertex.

Particles can also be done using transform feedback (Code can be almost be copied from Godot 3), but some features will not work like sub-emitters.

Support on Apple Devices

As OpenGL is being deprecated on Apple devices, we want to always use ANGLE (over Metal) for those platforms.

If this enhancement will not be used often, can it be worked around with a few lines of script?

N/A

Is there a reason why this should be core and not an add-on in the asset library?

N/A

[Calinou]

3D Rendering architecture for Godot 4.0 GLES3

Does this mean 3D support in OpenGL is planned for Godot 4.0 in the end (even in an "early access" form)?

High bit depth framebuffers are expensive on mobile and old hardware

Can R10G10B10A2 framebuffers be used in GLES3 to still have some kind of HDR rendering (like in Vulkan mobile)?

[reduz]

@Calinou

Does this mean 3D support in OpenGL is planned for Godot 4.0 in the end (even in an "early access" form)?

Its not planned but that depends on contributors willing to work on it.

Can R10G10B10A2 framebuffers be used in GLES3 to still have some kind of HDR rendering (like in Vulkan mobile)?

The problem with this is that there is not a ton you can do with this tonemapping wise because you start losing precision quickly. I think those framebuffers are more useful on HDR displays (common on phones) and maybe to have some extra precision you want to use for basic glow effects, but thats about it.

[clayjohn]

Some other points:
* We will render 2D and 3D to the same sRGB texture (unless using GLOW, DOF, etc.).
* In 3D, we will tonemap and convert to sRGB in the main draw call instead of in a post-process
edit: these are captured in the main proposal now

[Xrayez]

As OpenGL is being deprecated on Apple devices, we want to always use ANGLE (over Metal) for those platforms.

I have an experimental PR which incorporates ANGLE builds for UWP (success story), posting this because there's nobody working on this, so might be useful.

[kisg]

Is OpenGL on Apple platforms a valid use-case? Wouldn't the Vulkan based backends provide much better results with MoltenVK?

[Calinou]

Is OpenGL on Apple platforms a valid use-case? Wouldn't the Vulkan based backends provide much better results with MoltenVK?

While MoltenVK should be supported on nearly every iOS device in use now, the Vulkan mobile renderer has a different feature set than the planned OpenGL 3D renderer. You generally want to use the same rendering backend on all platforms to ensure consistency. Otherwise, you will have to compensate for the different appearance in your own code, and this can take a lot of time for a solo developer. This different appearance is typically due to the linear versus sRGB rendering, but it's not the only reason it can occur.

This also applies to GLES3 and GLES2 in 3.x: GLES3 is more viable to use on iOS compared to Android, but it looks different and has many features that don't work in GLES2. Therefore, even if you use GLES2 as a functional baseline, you'll still need to account for this visual difference depending on the target platform.

[clayjohn]

Now that #77496 is merged, this proposal is complete (hooray)! We still have more features coming for the Compatibility renderer, but they are not tracked by this proposal