Improve foliage instancing

[TokisanGames]

Update

First pass of an MMI implementation was merged in #340, improved in #402 and individual commits

Redesign of the data structure was laid out in #524 and implemented in #523.

This ticket will stay open to track these future improvements:

• Drag and drop into asset dock for meshes 99b64d7
• Automatically adjust height of foliage on sculpting Update instancer transforms #424
• Replace editor.cpp if instancer enable w/ add/subtract??
• Option to remove all mesh types
• Smaller MMI chunks 32-64/region
• Store transforms in a data structure instead of an MM and generate as needed.
• Store transforms in local space instead of global
• Use root node if MeshInstance3D
• LODs in scene files
• Apply transforms in mesh asset scene file
• Collision generation Waiting on WIP: Make collision generation dynamic #278
• Option to instance on all collision (eg grass on rocks)
• Create & destroy small MMI segments dynamically.
• Highlight MMI instances on the ground for the asset selected in the dock (temp swap override material), and/or add a picker
• Handle other meshes in scene file? - probably not. MMIs are for one mesh
• Improve generated grass: Meshless, vertex count, pointy, bend, gradient, wind
• AO/sample ground?
• Drag and drop reorganize in asset dock

---

Original

My idea for a paintable instancer is:

• Painting occurs on a splat/control map that can be saved in storage, but never goes in VRAM.
• MMI chunks are generated as the camera moves
• MMI chunks are regenerated if contained in the edited aabb of any terrain modifications

#324 and #278 need to be completed before this.

Reasoning:

• A control map based particle shader means placement around vertices with noise offsets. MMI nodes can be placed exactly where painted.
• Erasing instances is easy when painting on a control map. It's difficult to find and erase instances with an MMI, but regenerating a whole chunk while editing isn't a big deal.
• Particle shaders might be 2-4% faster, but will require yet another control map in VRAM. Fine if it's placement based on ground texture, but not if paintable. We're already consuming 4GB VRAM in OOTA and need to conserve.
• Particles can be frustum and distance culled. MMIs cannot, but with small enough chunks should be fine.
• MMIs seem to operate with Godot's autolod system (from what I see in SGT code). Haven't tested it, and don't know about particles.

---

Other Notes

The current control map format has 11 bits free, and most likely need 9 of them for paintable uv scale, slope, and rotation. 2 more could possibly be added to the base & overlay, or used for non-destructive layers.

Thus we probably need a second control map that could allocate foliage particles, water, and other items. However this other control map may not need to be in vram. It could be used to populate an MMI or a buffer filled with positions for a vertex shader.

Surprisingly, Godot's autolod apparently works with MMIs, according to SimpleGrassTextured, so this should be tested with MMIs and particle shaders.

Other solutions

• Hterrain approach uses splatmaps to store detail layers, then uses that data to generate MMI chunks

• SimpleGrassTextured appears to be an MMI painter. See https://github.com/IcterusGames/SimpleGrassTextured/blob/main/addons/simplegrasstextured/grass.gd.

• What do other solutions use? MTerrain, UE, Unity?

My ideas

• A paintable control map Image that does not get stored in vram. This is used to generate MMI chunks. Chunks are regenerated upon sculpting updates within the editable AABB.
• Use a vram control map that feeds directly into a particle shader. Offer 2 particle types per vertex and a (paintable?) density. Several instances of the specified particles are placed around the vertex based on sampled noise and density.
• Collision could be generated on enabled particles, especially with the chunk based generator.
• Compute based grass https://ueshita.hatenablog.jp/entry/2023/12/06/220451
• Either particle or MMI
• Should be paintable w/ multiple levels eg. paint grass, sunflowers, rocks
• Collision is not expected, but could be generated with our chunk system
• Maybe an option for the user to provide either a textured mesh, or just a 2D texture that gets applied to a generated quad

https://godotshaders.com/shader/wandering-clipmap-stylized-grass/
https://godotshaders.com/shader/wandering-clipmap-foliage-particle-process/

• Ghost of Tsushima style Grass
  https://www.youtube.com/watch?v=bp7REZBV4P4&t=2s
  https://www.youtube.com/watch?v=Ibe1JBF5i5Y
  https://www.gdcvault.com/play/1027033/Advanced-Graphics-Summit-Procedural-Grass

• Consider rendering foliage without meshes
  https://www.youtube.com/watch?v=R-bjXOEQyX8
  VERTEX_ID and mesh-less rendering godotengine/godot#19473
  ArrayMesh with flag USES_EMPTY_VERTEX_ARRAY still needs vertex array for rendering work godotengine/godot#83446

• GL Grass
  https://github.com/SomePineaple/GLGrass
  https://www.youtube.com/watch?v=2h5NX9tIdno

• https://gpuopen.com/learn/mesh_shaders/mesh_shaders-procedural_grass_rendering/

• https://www.youtube.com/watch?v=jw00MbIJcrk

• Ghost of tsushima style grass in Godot
  https://github.com/2Retr0/GodotGrass

[Saul2022]

Maybe to add up on the description it could be faded in the distance like the visibility ranges, and giving an ability to get replaced in the distance by another mesh/alpha card. It could be good to see how the roblox engine manages a high amount of grass with a small loss on performance with particle shader. https://youtu.be/luhAhuR_nw4 at first it seems there's only s region that uses grass and by the distance lods appear there. and also botw https://youtu.be/3eN3h6hV45s

[Saul2022]

Sounds pretty good how it coming, though something that should be taken into account are custom lods, where for example for short distance you can have mesh foliage and in longer distance a billboard mesh is used with less vertices. I think for lod system the mohsen foliage paint can be useful ( it allows various lods and are customizable if you want to check it out https://github.com/mohsenph69/Godot-MTerrain-plugin, though don’t remember if he uses MMI).

Edit, yep he uses multimesh based on his grass introduction video https://www.youtube.com/watch?v=w31Mrag05cc

Also wouldn’t the particles be even faster in 4.3 thanks to the acyclic render graph(gpu particles)?.

[TokisanGames]

it could be faded in the distance like the visibility ranges

Distance fade is handled in the material of the user's object. If using MMI chunks distant chunks won't be generated.

custom lods

We'll be lucky to get any LODS at all. Custom lods with MMI chunks means 2-4x the number of chunks and instances to manage. If autolods work, and I haven't tested them yet, great. There's a proposal or idea to connect custom lods into the built in system, so that's perhaps the best way to get custom lods in MMI. However, I will look at other systems to see what they do. Links to the specific section of code are appreciated.

Also wouldn’t the particles be even faster in 4.3 thanks to the acyclic render graph(gpu particles)?.

I don't know much about it. Do you have a link where it talks about particles?

[Saul2022]

custom lods
I will look at other systems to see what they do. Links to the specific section of code are appreciated.

Yea sounds understandable though i think for that you could talk with mohsen about his approach, which seem’s to work well. As for his system let’s see if i find the files where he uses it.

Edit: alright the part of the code that shows grass is at https://github.com/mohsenph69/Godot-MTerrain-plugin/tree/master/gdextension/src/grass

Where it seem’s to show the lod part is in lod sething h and gdextion, where it seem’s to store the lods and being used at the mgrass code, which could be included in the grass chunk code where it calcultates that part ( it what i think, i am no expert in this stuff and i can commit a lot of mistakes. ).

I don't know much about it. Do you have a link where it talks about particles?

Yes here it is about the performance diference when it talks results — gpu performance.

https://godotengine.org/article/rendering-acyclic-graph/

[TokisanGames]

Good discussion on #rendering godot dev chat. Bottom line, acyclic graph won't help fixed particles like grass much. MMI is simpler and more efficient than a particle shader (which uses MMI in the background).

Cory
7:56 PM
I'm deciding between a Particle Shader and Multimesh Instancing for a foliage instancer in Terrain3D and have some questions that I haven't been able to find out for myself in the code:
1. Does the new acyclic graph improve MMI like it does particles?
2. Do MMIs use particles in the background?
3. Does the data container that stores MMI transforms consume VRAM, as a splatmap/uniform array that defines placement for a particle shader would?
4. It appears that MMI manages autolods, so I don't have to setup multiple MMIs to manually swap LODs. (Haven't tested it, seems to be undocumented, but SimpleGrassTextured documentation shows it working). 
Is that true, and does that also work on particles? https://github.com/IcterusGames/SimpleGrassTextured#lod-optimization
5. Any other pros and cons or recommendations on one path vs the other? I'm quite familiar with using both, but lack the esoteric knowledge of the renderer.
I'd like to support 64-128 particle types, paintable on 16k and eventually up to 90k terrains, highest instancing performance, minimal VRAM, built in LOD support if possible.

dariosamo
8:39 PM
> Does the new acyclic graph improve MMI like it does particles?
No. MMI is reliant on a CPU-side structure you upload for the transforms. GPU particles benefited from the graph because the transforms are filled by the GPU instead (hence why it parallelized it better).

Cory
10:24 PM
So only the initial loading is faster with the graph, but once the particles (say grass) are in place, rendering is not any faster?
And are we talking about this speedup is for GPUParticles3D? Or a particle shader, which would likely get transforms from a texture map?

clayjohn
10:29 PM
For MMI, acyclic graph doesn't make them any faster, that's right.
The speed benefit for Particles comes from parallelization. So the speed benefits will vary depending on your content. Basically, before particle updates (so running the particles process shader) happened sequentially. 
After the acyclic graph they run in parallel. So if you have a lot of particle systems that don't fully saturate the GPU (i.e., there are only a few thousand particles in each GPUParticles node), then you should get a huge speedup.
If your particles use a complex shader and you have tens of thousands of particles per node, then you might not see a huge speedup

QbieShay
10:34 PM
I think MMI would be the better fit for terrain. Particles have their own compute pipeline set up
If your objects are just static then there's no benefit to the extra setup cost

clayjohn
10:38 PM
Following up on the rest of the questions:
1. Answered
2. No MMI's do not use particles in the background, Particles use MMI in the background. The big difference is that particles run the particle shader pass and always have color and custom in their data structure. 
3. MMI are more efficient to render as a result
4. Yes, the MMI buffer consumes ram. It is always the same or less than what GPUParticles consumes (especially if you are passing a splatmap as a uniform buffer)
The auto LOD for multimesh is kind of buggy. If it works for you, then that's great. HLOD is a much better fit though IMO so you can adjust the number of instances over distance as well
5. Like I said before MMI are more efficient to render than Particles. However, Particles are more efficient if you are doing updates to their data every frame (and those updates can be done completely on the GPU). 
Updating every instance transform every frame, for example, will be slow with MMI compared to using the GPU to update the instance transforms.

[guitarmike]

Adding collisions to objects can't come soon enough. We can't have people running through trees! Once that's in, I can move on from Proton Scatter.

Love the plug-in: you're doing God's work here!

[Saul2022]

Adding collisions to objects can't come soon enough.

It's fine , the devs are busy with other stuff rn that is pretty important too , in the meantime keep it with proton

[scottdavis]

Would be nice to have the ability to select more then one mesh for building forests with random trees or a field with grass and flowers

[vassembly]

Would be nice to have the ability to select more then one mesh for building forests with random trees or a field with grass and flowers

Would be cool to have presets where each mesh has it's own weight.

Also, big thanks to all devs and contributors for making this plugin.