WIP: Make collision generation dynamic

[lw64]

Admin edit:

• Should also address Update debug collision on operations #106
• Test Fatal error (Index p_index = 0 is out of bounds) when closing the game #341
• Test CodeGenerated collision has extra lip #328
• Test Show debug collision doesn't set collision layer/mask #329
• Test Enabling collision increases memory consumption each time the scene is reloaded. #397

---

Fixes #161

[TokisanGames]

If you play this and watch your process monitor, the game continually consumes more and more memory as the player moves around. All memory allocation needs to be moved to _build_collision and all freeing moved to _destroy_collision.

Though the memory allocation for debug collision was here, it was only created at scene start and wasn't a problem until the repeated creation and destruction. Now debug collision allocation can also be moved to build_collision.

[lw64]

I couldn't move the memory allocation fully to _build_collision, as it is not defined at build time, how many collision shapes need to be allocated. the distance is variable, and for example for 64.0 the shape count varies between 48 and 52 in my experience, depending on the camera position.

[lw64]

I made it now allocate new ones on demand, but that should only happen during the first _update_collision call.

[TokisanGames]

Alright, good job. I think once the memory management is moved to build collision and destroy collision it will be on par or better than a single 65x65 collision shape. Worse because it has to manage 50-60 shapes, but better because it only needs to update a few smaller shapes instead of all 65x65.

We're pretty close. Make the modifications, rebase and it should be finished or very close.
Thanks!

[lw64]

I am not entirely sure this avoids allocations. But I also don't see another method. the map_data gets assigned back later again.

[TokisanGames]

The allocations and deallocations we want to avoid in update_collision are memnew and memfree. Creating a PackedFloat32Array here is fine. This gets created on the stack which is designed for frequent, rapid creation/destruction. The other two allocate memory on the heap, which should not be as frequent.

[lw64]

Which other two? Dictionary?

[TokisanGames]

By other two, I mean the general usage of memnew and memfree. Not referring to lines specifically.

[lw64]

One thing I observed is that when you change the collision_mode from full to dynamic, the collision shapes aren't freed. But I guess its not a problem since this isn't changed at runtime. And in the editor you can just reopen the scene

[TokisanGames]

I couldn't move the memory allocation fully to _build_collision, as it is not defined at build time, how many collision shapes need to be allocated.

• Array.resize() will resize arrays dynamically.
  shapes_per_region is calculated in update_collision, but doesn't need to be, it can be calculated in build_collision.

• In game modes (physics server) are broken:

ERROR: Index p_index = 0 is out of bounds (shapes.size() = 0).
   at: set_shape (servers/physics_3d/godot_collision_object_3d.cpp:52)
ERROR: Index p_index = 0 is out of bounds (shapes.size() = 0).
   at: set_shape_transform (servers/physics_3d/godot_collision_object_3d.cpp:63)

• Three related issues don't rebuild collision properly. Upon set_collision_mode, set shape size, & distance it should destroy and build collision to free allocations and set everything up properly, just like set_collision_enabled does.

  • Changing from Full editor to Dynamic editor all collision meshes are huge and don't get resized down.

  • Changing shape size (on dynamic in editor mode) crashes godot.

  • Changing from an editor mode to a game mode does not remove the shapes, but should.

I didn't evaluate all of the new code since there are things that aren't working. Let me know if you need help with anything or want to work on things together.

I added a PR that renames and organizes some things.

[TokisanGames]

In dynamic editor mode, collision shapes do not have unique positions and are doubling up

---

Though this solution is an interesting method, I feel like this is an overly complicated solution.

What I proposed was using a small 65x65 collision square around the player, then on every snap, update the vertices of the shape, so 4225 vertex updates every snap, (which is based on movement, not frames). That is how my gterrain code worked, and is very short and simple. It will work fine with the static body and physics server, and does not continually allocate and deallocate memory.

This solution has improves upon it by breaking down the collision shape into 17x17 vertex chunks, and only new chunks are updated. So when moving laterally, say 8 chunks are updated, or 2312 vertices updated.

The cost of it though is a rather complicated chunking system that needs to track 50-64 collision shapes and move them around accurately and uniquely, without making memory non-contiguous by constant allocations and deallocations. This hasn't been achieved quite yet, so additional cost is that we need to spend more time to preallocate all of the shapes in build_collision, and identify unique positions so they don't doubling up.

Then we have to maintain this code over time.

And the physics server needs to calculate based on 64 shapes instead of 1.

Though I started working on this PR to clean up and fix things, once I discovered the doubling up, I think the scales tipped for me in the cost/benefit analysis.

I'm not sure the added overhead of us and the physics server managing all of the extra shapes, and the effort we've put in and still need to put in is worth the savings of ~2000 vertex assignments per snap. Whether it's 2000 or 4000 adjustments, both are inconsequential.

I think we should revert most of these changes and go back to the simpler solution proposed in #161 and demonstrated in my gterrain code. Though the editor collision modes and settable collision shape size are good.

What do you think?

[lw64]

So I addressed your issue with the doubled collision shapes and its fixed now. Also I finished the non-editor collision, which was previously not really done yet. The issues I had before with missing API was resolved by finding the correct API xD I tested this a bit and for me it doesn't leak any memory and there shouldn't be any allocations as far as I can see. So I think its good to go now.

[TokisanGames]

As soon as I switched to Dynamic / Editor running the demo crashes on this line. The camera is outside of the terrain. The reason is:
get_pixel is called with (896, 464), however cmap, cmap_x, cmap_xz, cmap_z are all null

Region is currently -1, which means no region. So if region == -1 after line 427, you can skip all of the code down to L558 probably. Though you may want to disable that shape.

That should allow shapes to be generated on the terrain if the player is near the sculpted regions, just not in the areas outside of that.

[TokisanGames]

Adding this in L428 is sufficient to have it work. However I haven't looked yet to see if this properly disables the shapes or if it needs to do more.

				if (region < 0) {
					continue;
				}

[TokisanGames]

I'm going to look at the design a bit more later, but here's some initial testing of functionality.

• It was working very well before, just not the ideal design. Now as it's being redesigned, the issues are normal. Just make sure to not squash your PR yet so we can refer back to commit d720d55 for reference.

• I recently realized the use of memfree() in _destroy_collision() is wrong. It should be memdelete() with memnew. memfree goes with memalloc. Changing these dramatically reduces the number of RID Allocations leaked at exit messages when godot closes, which it should. We weren't removing any of them.

• In Dynamic/editor mode, it seems to be generating only 1/4th of the circle so it doesn't work in game since it's off center.
  

• Full / editor mode only generates for the first region.

• On a couple of editor startups I got this error message. Other times it didn't happen. I didn't troubleshoot, so look out for it.

ERROR: Terrain3D::_update_collision: Collision not initialized, returning
   at: push_error (core/variant/variant_utility.cpp:1091)

[TokisanGames]

Note, I fixed the memory leaks for the up coming release.

c5115fe

[Saul2022]

An error i found checking this pr is that when trying to make play the demo, it doesn´t start and leaves the long messages, confirming cory issue.

that update coll is not initialized
It happen´s in both 4.2 and a 4.3 build.

either way , it improves loading times( like it the main reason why a game with terrain 3d takes a lot to load.

core\variant\variant_utility.cpp:1091 - Terrain3D::_update_collision: Collision not initialized, returning

[TokisanGames]

My current thinking for the foliage instancer is to generate MultiMeshes in chunks. I will build it off of the chunk infrastructure you're building here. I'll be ready to work on foliage in a week or two, so if this isn't done by then I'll get involved to move it along.

[TokisanGames]

This PR needs address #341 (comment), and make sure it works without error when running and quitting without regions.