Signal 11 crash with no debug info in backtrace

[htmiel]

Tested versions

Reproducible in: v4.3.beta2.official [b75f048], v4.3.beta1.official [a4f2ea9]

System information

Godot v4.3.beta2 - Windows 10.0.19045 - Vulkan (Forward+) - dedicated NVIDIA GeForce RTX 3060 (NVIDIA; 32.0.15.5585) - 13th Gen Intel(R) Core(TM) i9-13900K (32 Threads)

Issue description

I am working on a game and I want to test overall stability (this is my first Godot project). The task involves running game logic code continuously in a loop for ~10 seconds to see if any issues pop up. The code is "pure GDScript" (as it would run on the server app), meaning only code from custom classes; no nodes, no audio, nothing related to visuals, no physics, etc.

When testing this, the app occasionally freezes and crashes (both when running from editor, as well as exported), with CrashHandlerException: Program crashed with signal 11, and the rest of the info does not seem to help in identifying the problem:

CrashHandlerException: Program crashed with signal 11
Engine version: Godot Engine v4.3.beta2.official (b75f0485ba15951b87f1d9a2d8dd0fcd55e178e4)
Dumping the backtrace. Please include this when reporting the bug to the project developer.
[1] error(-1): no debug info in PE/COFF executable
[2] error(-1): no debug info in PE/COFF executable
[3] error(-1): no debug info in PE/COFF executable
[4] error(-1): no debug info in PE/COFF executable
[5] error(-1): no debug info in PE/COFF executable
[6] error(-1): no debug info in PE/COFF executable
[7] error(-1): no debug info in PE/COFF executable
[8] error(-1): no debug info in PE/COFF executable
-- END OF BACKTRACE --

What can I do to get more info about the source of the crash?
I attached the full log from running with --verbose: godot.log

I found https://github.com/Calinou/godot-debug-builds but it seems the last build is from a while back.

Steps to reproduce

I am not able to pinpoint the source of the crash.

Minimal reproduction project (MRP)

I cannot create an MRP as I do not know the source of the crash.

[kus04e4ek]

What can I do to get more info about the source of the crash?

Copied from here:

Currently, there are no official builds with debug symbols, so you'll have to compile from source with the debug_symbols=yes SCons option.

[kus04e4ek]

• Maybe related: 2D Physics Run on Seperate Thread Crash #93474

[htmiel]

Possibly also related to the fact that the editor itself sometimes crashes while using it (~ 1-2 times/day, when not running the game).

(currently building Godot debug, will return with results)

[htmiel]

I built Godot as debug with debug_symbols=yes, and I got the following files (no errors, all went smoothly thanks to the great docs):

After that, I launched the editor through console with --verbose, and tested until I got 4 crashes. Two of the times absolutely no errors were output in the log file, but two of the times I got this:

CrashHandlerException: Program crashed
Engine version: Godot Engine v4.3.beta.custom_build
Dumping the backtrace. Please include this when reporting the bug to the project developer.
[0] <couldn't map PC to fn name>
[1] <couldn't map PC to fn name>
[2] <couldn't map PC to fn name>
[3] <couldn't map PC to fn name>
...
[37] <couldn't map PC to fn name>
[38] <couldn't map PC to fn name>
[39] <couldn't map PC to fn name>
[40] <couldn't map PC to fn name>
-- END OF BACKTRACE --

Here is a full log (godot debug.log) where you can see multiple "simulations" that run and end correctly, until the last one starts before the crash.

What can I do to try and narrow down the cause of this crash?

[Calinou]

• Related to Distribute official builds with full debugging symbols godot-proposals#1342.

What can I do to try and narrow down the cause of this crash?

Try running Godot using the Visual Studio debugger or WinDbg, so that you don't need to rely on Godot's crash handler to see the backtrace.

[htmiel]

Try running Godot using the Visual Studio debugger

Question: after I run the Godot editor from Visual Studio with F5, and after that I run the game with F5 from the Godot editor, do I need to re-attach Visual Studio debugger to game process, or leave it on the editor process?

[htmiel]

After running the Godot editor from Visual Studio and performing the test 70 times (~1 hour of it running constantly), everything worked perfectly and I was unable to reproduce the crash...

Tomorrow I will build the editor and Windows templates separately, and run the test outside VS again.

[Calinou]

Question: after I run the Godot editor from Visual Studio with F5, and after that I run the game with F5 from the Godot editor, do I need to re-attach Visual Studio debugger to game process, or leave it on the editor process?

I don't remember if Visual Studio automatically attaches to the child process. I recommend passing command line arguments to the debugger so that the project runs directly instead.

[htmiel]

Thanks a lot guys for the guidance!

I managed to get some error info while testing a custom build with scons platform=windows debug_symbols=yes, and I got the following in the console/log (not launching with --verbose):

CrashHandlerException: Program crashed
Engine version: Godot Engine v4.3.beta.custom_build
Dumping the backtrace. Please include this when reporting the bug to the project developer.
[0] Variant::reference (Z:\godot\src\core\variant\variant.cpp:1090)
[1] GDScriptInstance::get (Z:\godot\src\modules\gdscript\gdscript.cpp:1730)
[2] Object::get (Z:\godot\src\core\object\object.cpp:310)
[3] Variant::get_named (Z:\godot\src\core\variant\variant_setget.cpp:314)
[4] GDScriptFunction::call (Z:\godot\src\modules\gdscript\gdscript_vm.cpp:1167)
[5] GDScriptInstance::callp (Z:\godot\src\modules\gdscript\gdscript.cpp:2028)
[6] Object::callp (Z:\godot\src\core\object\object.cpp:786)
[7] Variant::callp (Z:\godot\src\core\variant\variant_call.cpp:1211)
[8] GDScriptFunction::call (Z:\godot\src\modules\gdscript\gdscript_vm.cpp:1768)
[9] GDScriptInstance::callp (Z:\godot\src\modules\gdscript\gdscript.cpp:2028)
[10] Object::callp (Z:\godot\src\core\object\object.cpp:786)
[11] Variant::callp (Z:\godot\src\core\variant\variant_call.cpp:1211)
[12] GDScriptFunction::call (Z:\godot\src\modules\gdscript\gdscript_vm.cpp:1768)
[13] GDScriptInstance::callp (Z:\godot\src\modules\gdscript\gdscript.cpp:2028)
[14] Object::callp (Z:\godot\src\core\object\object.cpp:786)
[15] Variant::callp (Z:\godot\src\core\variant\variant_call.cpp:1211)
[16] GDScriptFunction::call (Z:\godot\src\modules\gdscript\gdscript_vm.cpp:1768)
[17] GDScriptLambdaSelfCallable::call (Z:\godot\src\modules\gdscript\gdscript_lambda_callable.cpp:275)
[18] Callable::callp (Z:\godot\src\core\variant\callable.cpp:58)
[19] Object::emit_signalp (Z:\godot\src\core\object\object.cpp:1190)
[20] Node::emit_signalp (Z:\godot\src\scene\main\node.cpp:3890)
[21] Object::emit_signal<> (Z:\godot\src\core\object\object.h:936)
[22] BaseButton::_pressed (Z:\godot\src\scene\gui\base_button.cpp:138)
[23] BaseButton::on_action_event (Z:\godot\src\scene\gui\base_button.cpp:176)
[24] BaseButton::gui_input (Z:\godot\src\scene\gui\base_button.cpp:69)
[25] Control::_call_gui_input (Z:\godot\src\scene\gui\control.cpp:1829)
[26] Viewport::_gui_call_input (Z:\godot\src\scene\main\viewport.cpp:1570)
[27] Viewport::_gui_input_event (Z:\godot\src\scene\main\viewport.cpp:1836)
[28] Viewport::push_input (Z:\godot\src\scene\main\viewport.cpp:3259)
[29] Window::_window_input (Z:\godot\src\scene\main\window.cpp:1690)
[30] CallableCustomMethodPointer<Window,Ref<InputEvent> const &>::call (Z:\godot\src\core\object\callable_method_pointer.h:103)
[31] Callable::callp (Z:\godot\src\core\variant\callable.cpp:58)
[32] Callable::call<Ref<InputEvent> > (Z:\godot\src\core\variant\variant.h:876)
[33] DisplayServerWindows::_dispatch_input_event (Z:\godot\src\platform\windows\display_server_windows.cpp:3510)
[34] Input::_parse_input_event_impl (Z:\godot\src\core\input\input.cpp:774)
[35] Input::flush_buffered_events (Z:\godot\src\core\input\input.cpp:1053)
[36] DisplayServerWindows::process_events (Z:\godot\src\platform\windows\display_server_windows.cpp:2978)
[37] OS_Windows::run (Z:\godot\src\platform\windows\os_windows.cpp:1686)
[38] widechar_main (Z:\godot\src\platform\windows\godot_windows.cpp:181)
[39] _main (Z:\godot\src\platform\windows\godot_windows.cpp:208)
[40] main (Z:\godot\src\platform\windows\godot_windows.cpp:220)
[41] __scrt_common_main_seh (D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288)
[42] <couldn't map PC to fn name>
-- END OF BACKTRACE --

Here is the full log: godot.log

You can see in it that the test ran 33 times before it crashed on attempt 34, but this is completely random, as it sometimes crashes on the first try as well.

[htmiel]

I mentioned in my original post that my test uses only "pure GDScript", but there is one other thing I actually use, and that is Image objects, as my game maps are stored as images so I can apply Image methods to them (get_pixel(), blend_rect(), create_from_data() and decompress_dynamic()).

Though I cannot see something related to images in the error info.

Regarding the editor crashes: does the Gogot editor itself save a log somewhere? I was unable to find one at the typical location where Godot projects keep their user files: %appdata%\Godot\

[Calinou]

Regarding the editor crashes: does the Gogot editor itself save a log somewhere? I was unable to find one at the typical location where Godot projects keep their user files: %appdata%\Godot\

It does not (unless you start it with --log-file path/to/file.log in 4.3 or later), but you can start the editor from a terminal and look at the output. On Windows, make sure to use the .console.exe wrapper executable.

[htmiel]

My main concern with this rare/random crashing would be not as much the game client itself, but having a server app that needs to run reliably for days/weeks (assuming no errors in my own code).

I'm not sure if this is relevant, but I also rarely get some random GDScript errors, like an "invalid access of a variable" that definitely exists, or an "invalid access of an array element" that is definitely there (I got this on an array declared directly in the script file and which is never modified). In the editor, the code would pause at such an error, but if I resume, it continues working fine. To me would appear that GDScript itself sometimes "glitches".

I will get exact error messages/screenshots the next time I see one of these.

[htmiel]

Here is an example of a random GDScript error that I previously mentioned. Hovering with the mouse over the cElement dictionary shows that the rotation property actually exists (and cSprite is a Sprite2D so it also has a rotation)

After resuming execution everything continued just fine.

[huwpascoe]

In that case, a dictionary needs to be accessed with braces:
cSprite.rotation = cElement["rotation"]
(this is incorrect)

[htmiel]

I noticed that both methods normally work, but I did not know that the braces method is better practice, thank you for this.

[huwpascoe]

I noticed that both methods normally work, but I did not know that the braces method is better practice, thank you for this.

Okay I'll admit I didn't know dictionaries could be accessed by that syntax. Would it be possible to share more of the code related to that dictionary? Rename types and things as needed if you don't want to share project details.

[htmiel]

Would it be possible to share more of the code related to that dictionary?

There is no special "code" about it, it's just a simple dictionary on which I set and read values from. When I started using Godot I was using the brackets syntax to set/get values (as I was used in the old Adobe AIR/Flash), but then I saw I can get/set values directly with dot syntax, which to me looks cleaner.

I love the dynamic typing of GDScript because I can focus on building my game instead of static-typing everything, and I'm totally fine with the small performance loss caused by it.

[huwpascoe]

(oh wow, adobe air that takes me back...)

What I meant is, if we can see where this dictionary is created, passed along, etc. that might be a clue to what's causing the bug.

[htmiel]

This particular dictionary is loaded from a .json on disk, parsed with JSON.parse_string(), and stored. As it describes a generic type of object in my game ("WorldObject" as visible in my screenshot, which extends Sprite2D), whenever I create such a new object instance I duplicate the dictionary with duplicate(true) and pass it to the object class, which does its thing with it, by continuously reading/changing values in it. I use the dot syntax, and I use the .has() method before accessing any properties that may or may not exist.

If the error would be caused by how I use the dictionary, I would expect it to show up more regularly, but instead it only happens randomly about once every few days (and my game regularly uses hundreds of these objects).

[huwpascoe]

That's good info, thanks.

[htmiel]

I got the new 4.3 beta 3 and tested again on 3 PCs (for a few hours). To my relief, it seems that only my PC experiences this rare, random, unreproducible crashing (but somehow not when running Godot from source with VS).

On one of the crashes I did get a new error I did not see before:

USER ERROR: Reaching unreachable case
   at: reduce_expression (modules/gdscript/gdscript_analyzer.cpp:2529)

As it would seem this is just an isolated issue, like in my case, it would explain the lack of other reports on this. For now I guess I can have some peace of mind to continue working in Godot knowing that it is indeed reliable :)

If you feel this issue is not worth any further investigation, please feel free to close it.

[RadiantUwU]

Due to what CPU it is, i need to know if it's the CPU itself or Godot.

I got the new 4.3 beta 3 and tested again on 3 PCs (for a few hours). To my relief, it seems that only my PC experiences this rare, random, unreproducible crashing (but somehow not when running Godot from source with VS).

This message makes it more look like this is the CPU and not Godot here.

[htmiel]

Mine is i9-13900K, and the others I tested on and worked without problems are: i7-8700K, i5-9300H, i5-4690T. Worth to note that my PC is pretty stable otherwise.

[RadiantUwU]

Mine is i9-13900K, and the others I tested on and worked without problems are: i7-8700K, i5-9300H, i5-4690T. Worth to note that my PC is pretty stable otherwise.

Are you underclocking currently?

[htmiel]

I did not make any changes to voltages, but the CPU has always been limited to 175W.

[404ar]

I think I have the same problem. Is there a solution?
`

CrashHandlerException: Program crashed with signal 11
Engine version: Godot Engine v4.3.stable.official (77dcf97)
Dumping the backtrace. Please include this when reporting the bug to the project developer.
[1] error(-1): no debug info in PE/COFF executable
[2] error(-1): no debug info in PE/COFF executable
[3] error(-1): no debug info in PE/COFF executable
[4] error(-1): no debug info in PE/COFF executable
[5] error(-1): no debug info in PE/COFF executable
[6] error(-1): no debug info in PE/COFF executable
[7] error(-1): no debug info in PE/COFF executable
[8] error(-1): no debug info in PE/COFF executable
[9] error(-1): no debug info in PE/COFF executable
[10] error(-1): no debug info in PE/COFF executable
[11] error(-1): no debug info in PE/COFF executable
[12] error(-1): no debug info in PE/COFF executable
[13] error(-1): no debug info in PE/COFF executable
[14] error(-1): no debug info in PE/COFF executable
[15] error(-1): no debug info in PE/COFF executable
[16] error(-1): no debug info in PE/COFF executable
[17] error(-1): no debug info in PE/COFF executable
[18] error(-1): no debug info in PE/COFF executable
[19] error(-1): no debug info in PE/COFF executable
-- END OF BACKTRACE --
================================================================

[htmiel]

I have concluded mostly that my problem is caused by the CPU which may be dying, since it's a 13900K and subject to the degradation issue I just recently found out about.

@404ar Do you have a high-power 13th/14th gen Intel CPU?

[404ar]

I have an old Intel® Celeron® Processor 2957U 2M Cache, 1.40 GHz i think that's the problem thank you @htmiel .

[404ar]

@htmiel I don't think the problem is with the processor, because I tried it on Linux and it worked without crashing , i think the problem is in the Windows system?

[htmiel]

I tested on a different CPU, and let a Godot Windows app run under high load for 2 days straight, no error. Your error could be either some isolated Windows issue, or something hardware related. But on a "healthy" Windows install and hardware, Godot worked great (at least with the Godot features I used).

If you can consistently reproduce the issue, you should try to isolate it as much as possible, to find out where it comes from, and them make a minimal reproduction project (MRP) that the contributors can investigate.

[htmiel]

I replaced my CPU today, and with all the tests I did, I had no more errors. This confirms that the errors I mentioned here were caused by my degraded Intel CPU.

[akien-mga]

Thanks for the update! Closing then.