Creating GridMaps from secondary thread throws octant_update errors

[SdgGames]

Godot version

v4.0.beta4.official [e675154]

System information

Windows Server 2022, Vulcan

Issue description

Creating a GridMap from a secondary thread throws errors and eventually causes a crash. This issue is in Godot 4, so I don't think that it's related to 42917

The GridMap is not a child of the scene tree when its contents are being modified. The following error is emitted while calling gridmap.set_cell_item

E 0:00:02:0592   _octant_update: Condition "!cell_map.has(E)" is true. Continuing.
  <C++ Source>   modules/gridmap/grid_map.cpp:610 @ _octant_update()

Additionally, the game will randomly crash at times.

Steps to reproduce

Start a secondary thread. From the secondary thread, create a GridMap. Do not add as a child of the scene yet.
Call set_cell_item() multiple times to populate the GridMap.
Errors will randomly appear, and the game might crash.

Using Reproduction Project:
Pressing the "Start" button will cause errors to show up in the debugging output
Pressing the "Start" button multiple times in will eventually cause the game to crash.

Minimal reproduction project

GridMap.zip

[Laserology]

A temporary fix exists by adding the items on the main thread:

YourGridMap.call_deferred("set_cell_item", Vector3i(X, Y, Z), ID)

This won't cause any noticeable slowdown on the main thread, as it is only adding the block. The other code that surrounds it will still be running on their own thread.

[diegopau]

My use case is a bit different and instead of trying to set GridMap cells in a separate thread I am instantiating a whole scene in a separate thread that contains several GridMaps (since I use each GridMap as a layer) among other nodes. I guess when you do so internally Godot is doing set_cell_item operations or similar to build GridMaps contained by the scene and it throws the exact same error as reported in the initial post in this issue.

So the fix from Laserology is not fully solving this for me.

[Laserology]

My use case is a bit different and instead of trying to set GridMap cells in a separate thread I am instantiating a whole scene in a separate thread that contains several GridMaps (since I use each GridMap as a layer) among other nodes. I guess when you do so internally Godot is doing set_cell_item operations or similar to build GridMaps contained by the scene and it throws the exact same error as reported in the initial post in this issue.

So the fix from Laserology is not fully solving this for me.

I personally do not have an issue with doing this, are you sure you aren't doing a cross-thread operation somewhere?

[diegopau]

You are totally right @Laserology Thank you for looking into this! I was about to completely change strategy for loading gridmap based levels in my game but thanks to your comment I looked into this more.
What I did is a very basic Godot project with a simple gridmap scene that can get loaded in 3 different ways (single thread, a separate thread without ResourceLoader, separate thread using ResourceLoader. It worked in the 3 ways. This is all I need to know to look into this again. I realize now that when I am instantiating those scenes in the secondary thread, that will execute code in scripts contained by those scenes, which probably is causing cross-thread operations.

In case it is helpful for anyone, I attach the simple project created that allows to test this.
simple-gridmap-project.zip

[diegopau]

I looked into this more and run more tests. The issue is indeed when instantiating the scene that contains GridMaps in a thread that is not the main thread. But it seems that the issue is related with the size of the GridMap and it won't fail with small GridMaps.

Large GridMaps are anyways giving Godot a hard time as it crashes the editor often by just moving the GridMap around in the scene with this error:

ERROR: FATAL: Index p_index = 4294967294 is out of bounds (count = 32771).
   at: operator[] (./core/templates/local_vector.h:177)

This is kind of random since sometimes I move it to were I want without a crash while other times it crashes.

So I might just be using GridMaps that are larger than the intended purpose of the engine. But it is a shame since other than those crashes in the editor or the problem I am reporting here with instancing in a secondary thread, the engine handles the rendering of those large GridMaps without issues in-game and with a decent performance.

To reproduce this without all the complexity of my main project I created a simple project that contains a very simple setup to test this.
simple-gridmap-project.zip

There you will find these options if you click in the main node:

By default it will try to load a scene with a large gridmap (a 128x128x128 gridmap full of solid cells) and in my case it does not load it correctly. Now the way this fails seems to vary in each run:

• Sometimes I had a crash
• Sometimes it did not crash but it showed these kind of errors:

E 0:00:03:0047   _octant_update: Condition "!cell_map.has(E)" is true. Continuing.
  <C++ Source>   modules/gridmap/grid_map.cpp:596 @ _octant_update()

• Sometimes however it will not crash or even throw errors, and yet I see small defects in the scene in the form of missing cells in the GridMap.
  This makes me think that depending on the computer of some users it might even just work fine since it seems to have random outcomes and this 128x128x128 gridmap seems to be almost there in the limit of what is able to handle.

However, if I choose the "Single Thread" option in that Main node, then it will always always load correctly.

When the scene loads you can move around using AWSD and the mouse (mouse wheel to change speed).

I might be doing something wrong with the way I load from a thread as I am more in the noob side for game development and Godot. The code is quite small so it should be easy to review.

Example of single thread load:

Example of threaded load (see the missing cells, but sometimes it is much less obvious than this):