Crash while CTRL+SHIFT tilemap on viewport

[Lagomorpheux]

Godot version

Godot Engine v4.0.2.stable.official.7a0977ce2

System information

Ubuntu 22.10, 64bit, NVIDIA GeForce GTX 1650, Vulkan API 1.3.224

Issue description

Godot crashes while painting a tilemap of a 900x900 pixels png texture while CTRL+SHIFT in the viewport.

Steps to reproduce

1. Create 2D node scene.
2. Add child of type TileMap.
3. On the inspector, create new TileSet.
4. On the TileSet tab, drag and drop a 900x900 px png image.
5. Click on yes to the atlas prompt.
6. On the TileMap tab, select the whole picture.
7. Zoom out the viewport so you have enough room for loads of tiles. In my case 5% is enough to reproduce the bug.
8. In the viewport, click somewhere on the top left corner while pressing CTRL and SHIFT.
9. Drag the rect to the bottom right.
10. Release the click. There I see the crash.

Minimal reproduction project

CrashDemo.zip

[komugi1211s]

Seems to be a stack overflow crash caused by the repeated call of alloca within the really long for loop (in my machine this line is called about 173876 times before causing a segfault down the load).

godot/core/object/undo_redo.cpp

Line 340 in 1becfea

const Variant **args = (const Variant **)alloca(sizeof(const Variant **) * binds.size());

[Calinou]

I can confirm this on 4.1.dev 33957ae (Linux). Note that the crash handler isn't called in this case, so you have to use gdb manually.

Backtrace:

#0  0x00007ffff7d7f28e in _int_malloc () from /lib64/libc.so.6
#1  0x00007ffff7d7fc34 in malloc () from /lib64/libc.so.6
#2  0x0000000004b14722 in Memory::alloc_static (p_pad_align=false, p_bytes=p_bytes@entry=32)
    at core/os/memory.cpp:75
#3  operator new (p_size=p_size@entry=32, p_description=p_description@entry=0x567e852 "") at core/os/memory.cpp:40
#4  0x00000000037b29f5 in DefaultTypedAllocator<HashMapElement<Vector2i, TileMapCell> >::new_allocation<HashMapElement<Vector2i, TileMapCell> > (this=0x1ca06e60) at ./core/os/memory.h:205
#5  HashMap<Vector2i, TileMapCell, HashMapHasherDefault, HashMapComparatorDefault<Vector2i>, DefaultTypedAllocator<HashMapElement<Vector2i, TileMapCell> > >::_insert (p_front_insert=<optimized out>, p_value=..., p_key=..., 
    this=<optimized out>) at ./core/templates/hash_map.h:223
#6  HashMap<Vector2i, TileMapCell, HashMapHasherDefault, HashMapComparatorDefault<Vector2i>, DefaultTypedAllocator<HashMapElement<Vector2i, TileMapCell> > >::insert (this=<optimized out>, p_key=..., p_value=..., 
    p_front_insert=false) at ./core/templates/hash_map.h:521
#7  0x00000000037993a5 in TileMap::set_cell (this=0x20c506e0, p_layer=0, p_coords=..., 
    p_source_id=<optimized out>, p_atlas_coords=..., p_alternative_tile=0) at scene/2d/tile_map.cpp:2154
#8  0x00000000037a83c3 in call_with_variant_args_helper<__UnexistingClass, int, Vector2i const&, int, Vector2i, int, 0ul, 1ul, 2ul, 3ul, 4ul> (r_error=..., p_args=<synthetic pointer>, p_method=<optimized out>, 
    p_instance=<optimized out>) at ./core/variant/binder_common.h:303
#9  call_with_variant_args_dv<__UnexistingClass, int, Vector2i const&, int, Vector2i, int> (default_values=..., 
    r_error=..., p_argcount=<optimized out>, p_args=<optimized out>, p_method=<optimized out>, 
    p_instance=<optimized out>) at ./core/variant/binder_common.h:450
#10 MethodBindT<int, Vector2i const&, int, Vector2i, int>::call (this=<optimized out>, p_object=<optimized out>, 
    p_args=<optimized out>, p_arg_count=<optimized out>, r_error=...) at ./core/object/method_bind.h:331
#11 0x000000000503d659 in Object::callp (this=0x20c506e0, p_method=..., p_args=0x7fffff7ff360, p_argcount=5, 
    r_error=...) at core/object/object.cpp:739
#12 0x0000000004da86d7 in Callable::callp (this=this@entry=0x304e0aa0, p_arguments=<optimized out>, 
    p_argcount=p_argcount@entry=5, r_return_value=..., r_call_error=...) at core/variant/callable.cpp:62
#13 0x0000000004dab1db in CallableCustomBind::call (this=0x304e0a90, p_arguments=<optimized out>, p_argcount=0, 
    r_return_value=..., r_call_error=...) at core/variant/callable_bind.cpp:144
#14 0x000000000507e16f in UndoRedo::_process_operation_list (this=this@entry=0x1ec68160, E=0x304e0ae0)
    at core/object/undo_redo.cpp:327
#15 0x000000000507fbcc in UndoRedo::_redo (this=this@entry=0x1ec68160, p_execute=p_execute@entry=true)
    at core/object/undo_redo.cpp:75
#16 0x0000000005081151 in UndoRedo::commit_action (p_execute=true, this=0x1ec68160)
    at core/object/undo_redo.cpp:306
#17 UndoRedo::commit_action (this=0x1ec68160, p_execute=p_execute@entry=true) at core/object/undo_redo.cpp:289
#18 0x000000000215053a in EditorUndoRedoManager::commit_action (this=this@entry=0xa9a2320, 
    p_execute=p_execute@entry=true) at editor/editor_undo_redo_manager.cpp:242
#19 0x0000000002d8a07a in TileMapEditorTilesPlugin::_stop_dragging (this=this@entry=0x7fff98079720)
    at editor/plugins/tiles/tile_map_editor.cpp:1432
#20 0x0000000002d92962 in TileMapEditorTilesPlugin::forward_canvas_gui_input (this=0x7fff98079720, p_event=...)
    at editor/plugins/tiles/tile_map_editor.cpp:706
#21 0x0000000001f726ed in EditorPluginList::forward_gui_input (this=0x198c8df0, p_event=...)
    at ./core/templates/cowdata.h:154
#22 0x00000000025395a6 in CanvasItemEditor::_gui_input_viewport (this=0xa904670, p_event=...)
    at editor/plugins/canvas_item_editor_plugin.cpp:2522
#23 0x0000000002547eea in call_with_variant_args_helper<CanvasItemEditor, Ref<InputEvent> const&, 0ul> (
    r_error=..., p_args=<optimized out>, p_method=<optimized out>, p_instance=<optimized out>)
    at ./core/variant/binder_common.h:303
#24 call_with_variant_args<CanvasItemEditor, Ref<InputEvent> const&> (r_error=..., p_argcount=<optimized out>, 
    p_args=<optimized out>, p_method=<optimized out>, p_instance=<optimized out>)
    at ./core/variant/binder_common.h:417
#25 CallableCustomMethodPointer<CanvasItemEditor, Ref<InputEvent> const&>::call (this=<optimized out>, 
    p_arguments=<optimized out>, p_argcount=<optimized out>, r_return_value=..., r_call_error=...)
    at ./core/object/callable_method_pointer.h:104
#26 0x00000000050474e8 in Object::emit_signalp (this=0x13e32cc0, p_name=..., p_args=0x7fffffffbe50, p_argcount=1)
    at core/object/object.cpp:1068
#27 0x000000000311ae56 in Object::emit_signal<Ref<InputEvent> > (p_name=..., this=0x13e32cc0)
    at ./core/object/object.h:891
#28 Control::_call_gui_input (this=0x13e32cc0, p_event=...) at scene/gui/control.cpp:1793
#29 0x0000000003007daa in Viewport::_gui_call_input (this=this@entry=0xae7c950, 
    p_control=p_control@entry=0x13e32cc0, p_input=...) at scene/main/viewport.cpp:1540
#30 0x000000000300df01 in Viewport::_gui_input_event (this=this@entry=0xae7c950, p_event=...)
    at scene/main/viewport.cpp:1812
#31 0x0000000003023d8c in Viewport::push_input (this=0xae7c950, p_event=..., p_local_coords=<optimized out>)
    at scene/main/viewport.cpp:2977
#32 0x000000000306c47a in call_with_variant_args_helper<Window, Ref<InputEvent> const&, 0ul> (r_error=..., 
    p_args=<optimized out>, p_method=<optimized out>, p_instance=<optimized out>)
    at ./core/variant/binder_common.h:303
#33 call_with_variant_args<Window, Ref<InputEvent> const&> (r_error=..., p_argcount=<optimized out>, 
    p_args=<optimized out>, p_method=<optimized out>, p_instance=<optimized out>)
    at ./core/variant/binder_common.h:417
#34 CallableCustomMethodPointer<Window, Ref<InputEvent> const&>::call (this=<optimized out>, 
    p_arguments=<optimized out>, p_argcount=<optimized out>, r_return_value=..., r_call_error=...)
    at ./core/object/callable_method_pointer.h:104
#35 0x0000000000ab18cd in DisplayServerX11::_dispatch_input_event (this=0x86d43c0, p_event=...)
    at platform/linuxbsd/x11/display_server_x11.cpp:3712
#36 0x0000000004d3c78b in Input::_parse_input_event_impl (this=this@entry=0x7faca80, p_event=..., 
    p_is_emulated=p_is_emulated@entry=false) at core/input/input.cpp:718
#37 0x0000000004d3f28a in Input::flush_buffered_events (this=0x7faca80) at core/input/input.cpp:976
#38 0x0000000000ab77f1 in DisplayServerX11::process_events (this=0x86d43c0)
--Type <RET> for more, q to quit, c to continue without paging--
    at platform/linuxbsd/x11/display_server_x11.cpp:4780
#39 0x0000000000a8f900 in OS_LinuxBSD::run (this=this@entry=0x7fffffffc550)
    at platform/linuxbsd/os_linuxbsd.cpp:900
#40 0x0000000000a812da in main (argc=<optimized out>, argv=0x7fffffffcb38)
    at platform/linuxbsd/godot_linuxbsd.cpp:74