Crash on exit in the GDScript module when debugging Godot

[Zylann]

Godot version

Godot 4 11e1c83

System information

Windows 10 64 bits NVIDIA GeForce GTX 1060

Issue description

Something happened in GDScript during the last weeks or so, such that now everytime I open and close my project, Godot crashes when unregistering the GDScript module. That wasn't happening before. This crash is visible when debugging Godot on Windows, using the debug runtime (which Godot is still not doing in debug builds). At the very least, it is inconvenient because it crashes on exit everytime, which disrupts debugging sessions. It appears to not reproduce without running Godot that way, but that might just be luck.

Here is the error:

Exception thrown at 0x00007FF6B55C4960 in godot.windows.editor.dev.x86_64.exe: 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF.
Exception thrown at 0x00007FFFB409CD29 in godot.windows.editor.dev.x86_64.exe: Microsoft C++ exception: std::__non_rtti_object at memory location 0x000000092DBFED18.

And the call stack, always the same:

issue_debug_notification(const wchar_t * const message) Line 28 (c:\Users\Marc\AppData\Local\Programs\Microsoft VS Code\minkernel\crts\ucrt\src\appcrt\internal\report_runtime_error.cpp:28)
__acrt_report_runtime_error(const wchar_t * message) Line 154 (c:\Users\Marc\AppData\Local\Programs\Microsoft VS Code\minkernel\crts\ucrt\src\appcrt\internal\report_runtime_error.cpp:154)
abort() Line 61 (c:\Users\Marc\AppData\Local\Programs\Microsoft VS Code\minkernel\crts\ucrt\src\appcrt\startup\abort.cpp:61)
terminate() Line 59 (c:\Users\Marc\AppData\Local\Programs\Microsoft VS Code\minkernel\crts\ucrt\src\appcrt\misc\terminate.cpp:59)
FindHandler<__FrameHandler4>(EHExceptionRecord * pExcept, unsigned __int64 * pRN, _CONTEXT * pContext, _xDISPATCHER_CONTEXT * pDC, FH4::FuncInfo4 * pFuncInfo, unsigned char recursive, int CatchDepth, unsigned __int64 * pMarkerRN) Line 781 (d:\a01\_work\12\s\src\vctools\crt\vcruntime\src\eh\frame.cpp:781)
__InternalCxxFrameHandler<__FrameHandler4>(EHExceptionRecord * pExcept, unsigned __int64 * pRN, _CONTEXT * pContext, _xDISPATCHER_CONTEXT * pDC, FH4::FuncInfo4 * pFuncInfo, int CatchDepth, unsigned __int64 * pMarkerRN, unsigned char recursive) Line 355 (d:\a01\_work\12\s\src\vctools\crt\vcruntime\src\eh\frame.cpp:355)
__CxxFrameHandler4(EHExceptionRecord * pExcept, unsigned __int64 RN, _CONTEXT * pContext, _xDISPATCHER_CONTEXT * pDC) Line 304 (d:\a01\_work\12\s\src\vctools\crt\vcruntime\src\eh\risctrnsctrl.cpp:304)
ntdll.dll!00007fffb66b241f() (Unknown Source:0)
ntdll.dll!00007fffb66614a4() (Unknown Source:0)
ntdll.dll!00007fffb66b0f4e() (Unknown Source:0)
KernelBase.dll!00007fffb409cd29() (Unknown Source:0)
_CxxThrowException(void * pExceptionObject, const _s__ThrowInfo * pThrowInfo) Line 75 (d:\a01\_work\12\s\src\vctools\crt\vcruntime\src\eh\throw.cpp:75)
__RTDynamicCast(void * inptr, long VfDelta, void * srcVoid, void * targetVoid, int isReference) Line 291 (d:\a01\_work\12\s\src\vctools\crt\vcruntime\src\eh\rtti.cpp:291)
Object::cast_to<GDScript>(Object * p_object) Line 751 (godot4_fork\core\object\object.h:751)
GDScript::_get_gdscript_from_variant(const Variant & p_variant) Line 1296 (godot4_fork\modules\gdscript\gdscript.cpp:1296)
GDScript::_get_dependencies(RBSet<GDScript *,Comparator<GDScript *>,DefaultAllocator> & p_dependencies, const GDScript * p_except) Line 1309 (godot4_fork\modules\gdscript\gdscript.cpp:1309)
GDScript::get_dependencies() Line 1156 (godot4_fork\modules\gdscript\gdscript.cpp:1156)
GDScript::get_inverted_dependencies() Line 1180 (godot4_fork\modules\gdscript\gdscript.cpp:1180)
GDScript::get_must_clear_dependencies() Line 1195 (godot4_fork\modules\gdscript\gdscript.cpp:1195)
GDScript::clear() Line 1413 (godot4_fork\modules\gdscript\gdscript.cpp:1413)
GDScript::~GDScript() Line 1480 (godot4_fork\modules\gdscript\gdscript.cpp:1480)
GDScript::`scalar deleting destructor'(unsigned int) (Unknown Source:0)
memdelete<GDScript>(GDScript * p_class) Line 112 (godot4_fork\core\os\memory.h:112)
Ref<GDScript>::unref() Line 223 (godot4_fork\core\object\ref_counted.h:223)
Ref<GDScript>::~Ref<GDScript>() Line 233 (godot4_fork\core\object\ref_counted.h:233)
KeyValue<String,Ref<GDScript>>::~KeyValue<String,Ref<GDScript>>() (Unknown Source:0)
HashMapElement<String,Ref<GDScript>>::~HashMapElement<String,Ref<GDScript>>() (Unknown Source:0)
HashMapElement<String,Ref<GDScript>>::`scalar deleting destructor'(unsigned int) (Unknown Source:0)
memdelete<HashMapElement<String,Ref<GDScript>>>(HashMapElement<String,Ref<GDScript>> * p_class) Line 112 (godot4_fork\core\os\memory.h:112)
DefaultTypedAllocator<HashMapElement<String,Ref<GDScript>>>::delete_allocation(HashMapElement<String,Ref<GDScript>> * p_allocation) Line 206 (godot4_fork\core\os\memory.h:206)
HashMap<String,Ref<GDScript>,HashMapHasherDefault,HashMapComparatorDefault<String>,DefaultTypedAllocator<HashMapElement<String,Ref<GDScript>>>>::clear() Line 266 (godot4_fork\core\templates\hash_map.h:266)
GDScriptCache::~GDScriptCache() Line 400 (godot4_fork\modules\gdscript\gdscript_cache.cpp:400)
GDScriptCache::`scalar deleting destructor'(unsigned int) (Unknown Source:0)
memdelete<GDScriptCache>(GDScriptCache * p_class) Line 112 (godot4_fork\core\os\memory.h:112)
uninitialize_gdscript_module(ModuleInitializationLevel p_level) Line 152 (godot4_fork\modules\gdscript\register_types.cpp:152)
uninitialize_modules(ModuleInitializationLevel p_level) Line 244 (godot4_fork\modules\register_module_types.gen.cpp:244)
Main::cleanup(bool p_force) Line 3343 (godot4_fork\main\main.cpp:3343)
widechar_main(int argc, wchar_t * * argv) Line 183 (godot4_fork\platform\windows\godot_windows.cpp:183)
_main() Line 203 (godot4_fork\platform\windows\godot_windows.cpp:203)
main(int argc, char * * argv) Line 217 (godot4_fork\platform\windows\godot_windows.cpp:217)
WinMain(HINSTANCE__ * hInstance, HINSTANCE__ * hPrevInstance, char * lpCmdLine, int nCmdShow) Line 231 (godot4_fork\platform\windows\godot_windows.cpp:231)
invoke_main() Line 107 (d:\a01\_work\12\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:107)
__scrt_common_main_seh() Line 288 (d:\a01\_work\12\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288)
__scrt_common_main() Line 331 (d:\a01\_work\12\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:331)
WinMainCRTStartup(void * __formal) Line 17 (d:\a01\_work\12\s\src\vctools\crt\vcstartup\src\startup\exe_winmain.cpp:17)
kernel32.dll!00007fffb64874b4() (Unknown Source:0)
ntdll.dll!00007fffb66626a1() (Unknown Source:0)

Steps to reproduce

Unfortunately this is a race condition, and I have no expertise in the GDScript module, so reproduction steps are far from clear. But I can give some details.

About the project:

• The project has no GDScript editor plugins
• Scenes and scripts use classes of a custom module, however I don't think it matters much
• It has no default scenes. The project opens on an empty scene by default.
• All it takes me to reproduce it is to open the project, and close it. Nothing else.
• However, the project is far from empty. It contains a bunch of testing stuff. Scripts, scenes, textures, models.
• Some of the scripts in the project have errors in them.
• The script editor has many scripts open from previous sessions, which are opening automatically when I open the project. I don't have to open the Script tab though.
  

About the Godot build:

• It includes my voxel module
• I build on Windows using the options scons p=windows target=editor dev_build=yes warnings=all werror=yes -j4
• In order to benefit from checks of the debug C++ runtime, I modified Godot's detect.py for Windows, as described in _DEBUG is not defined when compiling Godot in debug mode on Windows #31608
• Also described in _DEBUG is not defined when compiling Godot in debug mode on Windows #31608, in order to avoid another issue, you have to patch thirdparty/oidn/core/device.cpp in such a way that the thread_local inside is not created in global space. Just put it inside a static function that returns a reference to it.
• I run the editor with C++ debugger attached from VSCode

Minimal reproduction project

I was unable to reproduce this in a simpler project. It's likely a race condition, so it's quite random to stumble on the right combination that makes the crash occur. It may also depends on having opened scripts in a previous session, so opening it for the first time might not work. Despite my efforts to describe the context, it's possible that it doesn't reproduce for you. However if you run Godot the same way I did, you might stumble on a project that does.

project.zip

---

I can confirm it still crashes when no debugger is attached. I was profiling Godot using Tracy: when I closed Godot, it LOOKS like it closes normally, but Tracy picked up the crash.

[qarmin]

This is also visible with normal Godot build with address sanitizer(more info should be visible when using thread sanitizer )

=================================================================
==98750==ERROR: AddressSanitizer: heap-use-after-free on address 0x6040019853e0 at pc 0x000005df946f bp 0x7ffdef28c990 sp 0x7ffdef28c980
WRITE of size 8 at 0x6040019853e0 thread T0
    #0 0x5df946e in HashMap<String, Ref<GDScript>, HashMapHasherDefault, HashMapComparatorDefault<String>, DefaultTypedAllocator<HashMapElement<String, Ref<GDScript> > > >::erase(String const&) core/templates/hash_map.h:342
    #1 0x5deba57 in GDScriptCache::remove_script(String const&) modules/gdscript/gdscript_cache.cpp:176
    #2 0x5b9d10a in GDScript::clear() modules/gdscript/gdscript.cpp:1431
    #3 0x5b9f09e in GDScript::~GDScript() modules/gdscript/gdscript.cpp:1477
    #4 0x4094fc8 in void memdelete<GDScript>(GDScript*) core/os/memory.h:109
    #5 0x4059af7 in Ref<GDScript>::unref() core/object/ref_counted.h:221
    #6 0x3f9ff45 in Ref<GDScript>::~Ref() core/object/ref_counted.h:233
    #7 0x5c528bd in KeyValue<String, Ref<GDScript> >::~KeyValue() core/templates/pair.h:82
    #8 0x5c529db in HashMapElement<String, Ref<GDScript> >::~HashMapElement() core/templates/hash_map.h:55
    #9 0x5e11165 in void memdelete<HashMapElement<String, Ref<GDScript> > >(HashMapElement<String, Ref<GDScript> >*) core/os/memory.h:109
    #10 0x5e06dd7 in DefaultTypedAllocator<HashMapElement<String, Ref<GDScript> > >::delete_allocation(HashMapElement<String, Ref<GDScript> >*) core/os/memory.h:206
    #11 0x5e03100 in HashMap<String, Ref<GDScript>, HashMapHasherDefault, HashMapComparatorDefault<String>, DefaultTypedAllocator<HashMapElement<String, Ref<GDScript> > > >::clear() core/templates/hash_map.h:265
    #12 0x5df3c25 in GDScriptCache::~GDScriptCache() modules/gdscript/gdscript_cache.cpp:429
    #13 0x62a7512 in void memdelete<GDScriptCache>(GDScriptCache*) (/usr/bin/godot4s+0x62a7512)
    #14 0x628e883 in uninitialize_gdscript_module(ModuleInitializationLevel) modules/gdscript/register_types.cpp:149
    #15 0x41f0896 in uninitialize_modules(ModuleInitializationLevel) modules/register_module_types.gen.cpp:233
    #16 0x28d9840 in Main::cleanup(bool) main/main.cpp:3342
    #17 0x26c820e in main platform/linuxbsd/godot_linuxbsd.cpp:75
    #18 0x7f8d851a6d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #19 0x7f8d851a6e3f in __libc_start_main_impl ../csu/libc-start.c:392
    #20 0x26c7bb4 in _start (/usr/bin/godot4s+0x26c7bb4)

0x6040019853e0 is located 16 bytes inside of 48-byte region [0x6040019853d0,0x604001985400)
freed by thread T0 here:
    #0 0x7f8d85d0b517 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:127
    #1 0x1d21de9f in Memory::free_static(void*, bool) core/os/memory.cpp:168
    #2 0x5e11176 in void memdelete<HashMapElement<String, Ref<GDScript> > >(HashMapElement<String, Ref<GDScript> >*) core/os/memory.h:112
    #3 0x5e06dd7 in DefaultTypedAllocator<HashMapElement<String, Ref<GDScript> > >::delete_allocation(HashMapElement<String, Ref<GDScript> >*) core/os/memory.h:206
    #4 0x5e03100 in HashMap<String, Ref<GDScript>, HashMapHasherDefault, HashMapComparatorDefault<String>, DefaultTypedAllocator<HashMapElement<String, Ref<GDScript> > > >::clear() core/templates/hash_map.h:265
    #5 0x5df3c25 in GDScriptCache::~GDScriptCache() modules/gdscript/gdscript_cache.cpp:429
    #6 0x62a7512 in void memdelete<GDScriptCache>(GDScriptCache*) (/usr/bin/godot4s+0x62a7512)
    #7 0x628e883 in uninitialize_gdscript_module(ModuleInitializationLevel) modules/gdscript/register_types.cpp:149
    #8 0x41f0896 in uninitialize_modules(ModuleInitializationLevel) modules/register_module_types.gen.cpp:233
    #9 0x28d9840 in Main::cleanup(bool) main/main.cpp:3342
    #10 0x26c820e in main platform/linuxbsd/godot_linuxbsd.cpp:75
    #11 0x7f8d851a6d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

previously allocated by thread T21 here:
    #0 0x7f8d85d0b867 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x1d21cdc1 in Memory::alloc_static(unsigned long, bool) core/os/memory.cpp:75
    #2 0x1d21ccd2 in operator new(unsigned long, char const*) core/os/memory.cpp:40
    #3 0x5c92634 in HashMapElement<String, Ref<GDScript> >* DefaultTypedAllocator<HashMapElement<String, Ref<GDScript> > >::new_allocation<HashMapElement<String, Ref<GDScript> > >(HashMapElement<String, Ref<GDScript> > const&&) core/os/memory.h:205
    #4 0x5c53964 in HashMap<String, Ref<GDScript>, HashMapHasherDefault, HashMapComparatorDefault<String>, DefaultTypedAllocator<HashMapElement<String, Ref<GDScript> > > >::_insert(String const&, Ref<GDScript> const&, bool) core/templates/hash_map.h:223
    #5 0x5c116a6 in HashMap<String, Ref<GDScript>, HashMapHasherDefault, HashMapComparatorDefault<String>, DefaultTypedAllocator<HashMapElement<String, Ref<GDScript> > > >::operator[](String const&) core/templates/hash_map.h:512
    #6 0x5def226 in GDScriptCache::get_full_script(String const&, Error&, String const&, bool) modules/gdscript/gdscript_cache.cpp:289
    #7 0x5bc65a7 in ResourceFormatLoaderGDScript::load(String const&, String const&, Error*, bool, float*, ResourceFormatLoader::CacheMode) modules/gdscript/gdscript.cpp:2613
    #8 0x1da35eb7 in ResourceLoader::_load(String const&, String const&, String const&, ResourceFormatLoader::CacheMode, Error*, bool, float*) core/io/resource_loader.cpp:206
    #9 0x1da37473 in ResourceLoader::_thread_load_function(void*) core/io/resource_loader.cpp:233
    #10 0x1da3dd8f in ResourceLoader::load(String const&, String const&, ResourceFormatLoader::CacheMode, Error*) core/io/resource_loader.cpp:561
    #11 0xcf6b6f1 in EditorFileSystem::_scan_new_dir(EditorFileSystemDirectory*, Ref<DirAccess>&, EditorFileSystem::ScanProgress const&) editor/editor_file_system.cpp:922
    #12 0xcf660ba in EditorFileSystem::_scan_new_dir(EditorFileSystemDirectory*, Ref<DirAccess>&, EditorFileSystem::ScanProgress const&) editor/editor_file_system.cpp:791
    #13 0xcf555ce in EditorFileSystem::_scan_filesystem() editor/editor_file_system.cpp:320
    #14 0xcf571dc in EditorFileSystem::_thread_func(void*) editor/editor_file_system.cpp:346
    #15 0x1d22f6e1 in Thread::callback(Thread*, Thread::Settings const&, void (*)(void*), void*) core/os/thread.cpp:65
    #16 0x1d2336d5 in void std::__invoke_impl<void, void (*)(Thread*, Thread::Settings const&, void (*)(void*), void*), Thread*, Thread::Settings, void (*)(void*), void*>(std::__invoke_other, void (*&&)(Thread*, Thread::Settings const&, void (*)(void*), void*), Thread*&&, Thread::Settings&&, void (*&&)(void*), void*&&) /usr/include/c++/11/bits/invoke.h:61
    #17 0x1d2332e7 in std::__invoke_result<void (*)(Thread*, Thread::Settings const&, void (*)(void*), void*), Thread*, Thread::Settings, void (*)(void*), void*>::type std::__invoke<void (*)(Thread*, Thread::Settings const&, void (*)(void*), void*), Thread*, Thread::Settings, void (*)(void*), void*>(void (*&&)(Thread*, Thread::Settings const&, void (*)(void*), void*), Thread*&&, Thread::Settings&&, void (*&&)(void*), void*&&) /usr/include/c++/11/bits/invoke.h:96
    #18 0x1d232d57 in void std::thread::_Invoker<std::tuple<void (*)(Thread*, Thread::Settings const&, void (*)(void*), void*), Thread*, Thread::Settings, void (*)(void*), void*> >::_M_invoke<0ul, 1ul, 2ul, 3ul, 4ul>(std::_Index_tuple<0ul, 1ul, 2ul, 3ul, 4ul>) /usr/include/c++/11/bits/std_thread.h:253
    #19 0x1d2329b1 in std::thread::_Invoker<std::tuple<void (*)(Thread*, Thread::Settings const&, void (*)(void*), void*), Thread*, Thread::Settings, void (*)(void*), void*> >::operator()() /usr/include/c++/11/bits/std_thread.h:260
    #20 0x1d232965 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(Thread*, Thread::Settings const&, void (*)(void*), void*), Thread*, Thread::Settings, void (*)(void*), void*> > >::_M_run() /usr/include/c++/11/bits/std_thread.h:211
    #21 0x2044c353 in execute_native_thread_routine (/usr/bin/godot4s+0x2044c353)

Thread T21 created by T0 here:
    #0 0x7f8d85caf685 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
    #1 0x2044c428 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/usr/bin/godot4s+0x2044c428)
    #2 0x1d22fbfd in Thread::start(void (*)(void*), void*, Thread::Settings const&) core/os/thread.cpp:82
    #3 0xcf63b78 in EditorFileSystem::scan() editor/editor_file_system.cpp:716
    #4 0xd4ae2ed in EditorNode::_begin_first_scan() editor/editor_node.cpp:4387
    #5 0xd6d19d3 in void call_with_variant_args_helper<EditorNode>(EditorNode*, void (EditorNode::*)(), Variant const**, Callable::CallError&, IndexSequence<>) (/usr/bin/godot4s+0xd6d19d3)
    #6 0xd6c8345 in void call_with_variant_args<EditorNode>(EditorNode*, void (EditorNode::*)(), Variant const**, int, Callable::CallError&) (/usr/bin/godot4s+0xd6c8345)
    #7 0xd6bc156 in CallableCustomMethodPointer<EditorNode>::call(Variant const**, int, Variant&, Callable::CallError&) const (/usr/bin/godot4s+0xd6bc156)
    #8 0x1de25a30 in Callable::callp(Variant const**, int, Variant&, Callable::CallError&) const core/variant/callable.cpp:50
    #9 0x1e9e83b9 in MessageQueue::_call_function(Callable const&, Variant const*, int, bool) core/object/message_queue.cpp:229
    #10 0x1e9e94ac in MessageQueue::flush() core/object/message_queue.cpp:275
    #11 0x13871429 in SceneTree::physics_process(double) scene/main/scene_tree.cpp:433
    #12 0x28d35b6 in Main::iteration() main/main.cpp:3141
    #13 0x26ec483 in OS_LinuxBSD::run() platform/linuxbsd/os_linuxbsd.cpp:799
    #14 0x26c8204 in main platform/linuxbsd/godot_linuxbsd.cpp:73
    #15 0x7f8d851a6d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

SUMMARY: AddressSanitizer: heap-use-after-free core/templates/hash_map.h:342 in HashMap<String, Ref<GDScript>, HashMapHasherDefault, HashMapComparatorDefault<String>, DefaultTypedAllocator<HashMapElement<String, Ref<GDScript> > > >::erase(String const&)

[anvilfolk]

@adamscott looks related to #67714 since the stacktrace is going through the must_clear functions?

[adamscott]

My PR #69865 should be able to fix this issue.