Crazy memory usage and crashes when importing prefabs

[JouBqa]

other files imported fine but prefab importing crashes godot

[JouBqa]

"a new protection page cannot be created for the stack"

[JouBqa]

I tried save everything to text files but it still crashed. What can I try?

[lyuma]

Unfortunately, Unidot is extremely memory hungry because Godot loads all textures and meshes into memory while importing: if the prefabs reference a lot of data, it might take up your RAM.

I have found 12-16GB is sometimes needed on some heavier assets.

saving as text format is not going to save any memory: it will be slower at serializing large assets such as animations.

my only suggestion is to try restarting godot before importing prefabs/scenes to clear up leaked memory, and then try importing one prefab and see if it works if you do a few at a time.

[JouBqa]

I tried importing one single prefab (a character model). I have 16gb RAM. I have restarted Godot. Anything else I can try?
I used https://github.com/barcoderdev/FBX2glTF/actions/
Should I use this one? https://github.com/godotengine/FBX2glTF/releases

[lyuma]

Can you share the .prefab file which causes the problem, as well as a copy of your unidot_asset_database.res (or the whole package if that is possible)

If you want to PM me, my username on discord is lyuma, or you can email me xnlyuma@gmail.com

[JouBqa]

emailed you

[JouBqa]

I guess no fix?

[lyuma]

Thanks for following up.

I have run it on the master branch version on Windows in a large project with a baseline memory usage of 2GB. When importing the synty dungeon package you mentioned, I monitored the memory usage and the working set peaked around 4.5 GB and commit size peaked around 6GB.

this is an extremely far cry from what you reported, so I need to do more work to try to replicate the conditions you hit.

For one thing, godot 4.3 master branch uses fbx while godot 4.2 converts models to gltf.

Even though I couldn't reproduce the issue you hit, I have a few ideas that may guarantee memory usage will remain far lower for textures and meshes: basically injecting small 1x1 textures or tiny meshes into the ResourceCache while importing prefabs and so on. It will take some time to implement but I will try and work on this.

there is another memory bottleneck which won't affect this specific package, but the way we parse .anim animations is extremely memory heavy since we parse every keyframe into a huge nested dictionary.

Godot 4.3 dev6 will be coming out in a few days (warning: dev5 is buggy/not compatible!). When that happens, feel free to try the got master version of unidot with that version of godot. It will hopefully perform better and replicate my results. Do note that once a project is upgraded to 4.3, you can't go back. So make a backup or use a new project.

long story short, I have ideas to improve unidot, and I can support it better once 4.3 hits beta because we will not need the fbx2gltf translator.

[JouBqa]

Thanks. I'll try with 4.3 dev6

[lyuma]

@JouBqa 4.3dev6 is out.
https://godotengine.org/article/dev-snapshot-godot-4-3-dev-6/
this version comes with native .fbx support and the latest git version of Unidot will use that.

also, would it be possible to open task manager or top and see how much memory godot is using before and after the import?