Add an option to update shadow maps less often

[Calinou]

master version of #54516.

This can be used to improve performance on low-end setups, at the cost of shadows visibly lagging behind for dynamic lights/objects when up close.

Unlike the 3.x implementation, this one appears to work better with the default shadow bias settings when point lights are involved.

Testing project: test_animated_shadows_master.zip

[reduz]

This is a known technique, but it probably needs a bit more customization. For directional lights, what you often do is update last cascade every 4 frames and the previous to last one every 2. For point and directional lights, we already divide those in 4 quadrants. We should set the update frequency for each quadrant, with the tiny ones having less and the bigger ones having more.

[reduz]

Additionally, with positional lights, if a quadrant updates every 4 frames, we should not update all together at the 4 frame intervals but staircase them so frame timing remains smooth. Doing this properly is a bit of a challenge.

[mrjustaguy]

Here's my 2 cents on this matter:

I Wouldn't try to mix General and Directional Shadow Updates, It'd be needlessly complex, with far too few gains, so I'd do them Separately, especially considering that Directional Shadow Processing is often Much more demanding compared to General Shadows from my experience.

Directional Lights Update Modes

0. Updates All, every Frame
1. For 4 Splits (most common use case) Render 2 Splits every frame (Split 1 Always, 2,3,4 one after the other) this would result in Performance similar to 2 Splits being rendered, but the visual quality of 4 Splits, albeit with a little lag on distant splits, however this won't be too observable in most cases even on lower FPS, as 1st split is the one that matters the most in terms of update frequency.
2. For 2 Splits Render 1 Split every frame (like for 2,3,4 in the above) this would result in Performance similar to Orthogonal being rendered, but the visual quality of 2 Splits, albeit with the shadows being rendered at half the framerate, which on lower FPS will be clear
3. For Orthogonal Doesn't make sense as you will only be creating a stuttering mess in terms of frame time consistency

General Lights Update Modes

0. Updates All Shadow Map Quadrants, every Frame
1. Updates 2 Shadow Map Quadrants a frame switching them every frame, will result in less obvious Shadow Lag
2. Updates 1 Shadow Map Quadrant a frame switching them every frame, will result in obvious Shadow Lag
3. Updates 1 or 2 Shadow Map Quadrants every frame, the rest render switching each frame, will result in Shadow Lag for Shadows of lower importance

Default settings I'd use (this includes Option 0, aka current behavior):
DL mode 1 - 4 Splits Quality for 2 Splits worth of rendering costs at the cost of Shadow Lag in the distance which will be hard to observe in 99% of the cases, especially at 60 FPS+ (considering most things that have visible Directional shadows beyond 1st split don't move much or have their shadows significantly change frame to frame)
GL mode 3 1,3 setup - Many Shadows are difficult to follow and observe Shadow Lag, so only the first few most significant lights need their shadows to sync up well in most cases, and at 60 FPS, a 50ms update time for secondary light shadows shouldn't be that clear, however at 30 FPS it will be easier to spot, so mode 3 2,2 setup or mode 1 would be better for those cases.

I don't see too much improvement in Performance from General Lights being staggered, especially as they can be occlusion culled, while Directional Lights would be a Significant savings in terms of performance for 4 splits, as you'd only do 2 passes instead of 4 per frame, and it'd often be preferable to have 4 splits worth of quality with a little lag for 2 splits worth of speed to having to actually use 2 splits for performance, keep in mind that Occlusion Culling doesn't affect Directional Shadow Casters because the Directional Light is always rendered, while General Lights themselves can be culled, and thus Reducing Computation in general for Directional Shadows is far more important.

[clayjohn]

Updates All Shadow Map Quadrants, every Frame
Updates 2 Shadow Map Quadrants a frame switching them every frame, will result in less obvious Shadow Lag
Updates 1 Shadow Map Quadrant a frame switching them every frame, will result in obvious Shadow Lag
Updates 1 or 2 Shadow Map Quadrants every frame, the rest render switching each frame, will result in Shadow Lag for Shadows of lower importance

To add to this, it will need to be slightly more complex as lights move between quadrants depending on their priority. By default the closest light takes up quadrant 1 and the next few lights fill quadrant two. So the risk in not updating every frame is not that the shadows will be slightly out of date, but that a light will render the shadows from a different light.

[mrjustaguy]

Honestly, I doubt General shadows in general would have any significant benefits, especially considering that afaik only shadows where something changed need to be re-rendered, and even if that were not the case, they don't have Nearly the same degree of processing required as only meshes within their AABBs get processed, and as they tend to be fairly small, they're small fish compared to Directional Shadows.

I'd personally split the two into separate PRs and focus on Directional Shadows as they're easier to do, and would offer the most significant benefits which are guaranteed to be sizable in complex scenes, not to mention fairly safe from nasty bugs popping up (as opposed to General Shadows due to their complexity), and make a simple General Shadow PR that'd wouldn't take these things into consideration too much, but would be used to see to see how much of a difference it'd actually make performance wise in test scenes, to see if it's worth even dealing with the logic.

Also one more consideration, Shadows that Lag could produce self shadowing on moving objects, which would be much more visible for general lights due to their much higher relative resolutions, while Directional Lights tend to be more forgiving in this regard.
I know this as I'm using extremely aggressively low Poly Shadow Meshes (about 10% or less of the original Mesh) and using Shadow Bias as my friend, Improving the Performance Significantly with next to no observable Quality Loss, and I can tell you, Spot Lights are the First to see Self Shadowing problems, Directional Lights are always the last, and even then it's mainly the first split due to it's fairly high Resolution, which would be updated every Frame with the most useful of the settings.

[Zireael07]

+1 to focusing on directional lights first

[mrjustaguy]

Hmm, One thing I don't think was solved here, If shadows update less often, and an object is moving, the object could get a ton of self-shadowing artifacts, because essentially it, and it's effective shadow mesh, are at two very different positions.

This could be solved by applying the shadows at a "staggered" rate as well, so having the shadows apply to Meshes according to where they were when the shadow was rendered and not by where they are now, but to do so, you'd need to keep past several transforms for each shadow casting object.

[Zireael07]

Or maybe we could have static vs dynamic objects like other engines do ;)

[mrjustaguy]

I think that'd be an over-complicated solution to this problem and doesn't address the root of the problem.
I don't think that keeping what, up to 8x (one for each quadrant and each split if all shadows are staggered) extra transforms per shadow receiving object would be that big of a hit to memory consumption or extra computation, and this self shadowing issue is likely to make this unusable for most users without such a mitigation.

[Calinou]

Hmm, One thing I don't think was solved here, If shadows update less often, and an object is moving, the object could get a ton of self-shadowing artifacts, because essentially it, and it's effective shadow mesh, are at two very different positions.

This is indeed something you can see in the 3.x version of this PR (unless you tweak light parameters to use my recommendations). Howeve, the master version of this PR doesn't suffer from this issue much.

[mrjustaguy]

Actually, it fundamentally does, and it's really, really easy to do.
Easy example: have a cube that is getting and casting shadows move in the direction of the Directional Light with staggered shadows. it's speed just needs to be enough to be fast enough to get through the bias, which really isn't that fast

There are a ton of times where this effect will be observable in practice, as any time you have something moving to it's shadows even a little, this will happen.

[Calinou]

• Superseded by Stepped update of directional shadow maps #76291.