stack-buffer-overflow related to BVH

[qarmin]

Godot version

Godot 3.4 beta 3

System information

Ubuntu 21.04, Intel HD 3000

Issue description

When using asan and ubsan and running minimal project, then after ~1/2 seconds I have this buffer overflow

core/math/bvh_split.inc:74:35: runtime error: index 2 out of bounds for type 'float [2]'
core/math/bvh_split.inc:74:62: runtime error: index 2 out of bounds for type 'float [2]'
core/math/bvh_split.inc:74:62: runtime error: load of address 0x7ffdae73dff8 with insufficient space for an object of type 'real_t'
0x7ffdae73dff8: note: pointer points here
 34 4a 19 42  2a 00 00 00 00 00 00 00  90 2b 61 cd b5 7f 00 00  03 df 3e cf b5 7f 00 00  f2 e6 1d 44
              ^ 
=================================================================
==31967==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffdae73dff8 at pc 0x0000113ae7db bp 0x7ffdae73df30 sp 0x7ffdae73df20
READ of size 4 at 0x7ffdae73dff8 thread T0
    #0 0x113ae7da in BVH_Tree<CollisionObject2DSW, 2, 128, true, Rect2, Vector2>::_split_leaf_sort_groups_simple(int&, int&, unsigned short*, unsigned short*, BVH_ABB<Rect2, Vector2> const*, BVH_ABB<Rect2, Vector2>) core/math/bvh_split.inc:74
    #1 0x113a5f7e in BVH_Tree<CollisionObject2DSW, 2, 128, true, Rect2, Vector2>::split_leaf_complex(unsigned int, BVH_ABB<Rect2, Vector2> const&) core/math/bvh_split.inc:252
    #2 0x11394812 in BVH_Tree<CollisionObject2DSW, 2, 128, true, Rect2, Vector2>::split_leaf(unsigned int, BVH_ABB<Rect2, Vector2> const&) core/math/bvh_split.inc:186
    #3 0x11388134 in BVH_Tree<CollisionObject2DSW, 2, 128, true, Rect2, Vector2>::_logic_choose_item_add_node(unsigned int, BVH_ABB<Rect2, Vector2> const&) core/math/bvh_logic.inc:212
    #4 0x1138c758 in BVH_Tree<CollisionObject2DSW, 2, 128, true, Rect2, Vector2>::item_set_pairable(BVHHandle const&, bool, unsigned int, unsigned int) core/math/bvh_public.inc:323
    #5 0x113831da in BVH_Manager<CollisionObject2DSW, true, 128, Rect2, Vector2, true>::set_pairable(BVHHandle const&, bool, unsigned int, unsigned int, bool) core/math/bvh.h:304
    #6 0x1137dc9a in BVH_Manager<CollisionObject2DSW, true, 128, Rect2, Vector2, true>::set_pairable(unsigned int, bool, unsigned int, unsigned int, bool) core/math/bvh.h:172
    #7 0x1137a2e4 in BroadPhase2DBVH::set_static(unsigned int, bool) servers/physics_2d/broad_phase_2d_bvh.cpp:46
    #8 0x113ed44f in CollisionObject2DSW::_set_static(bool) servers/physics_2d/collision_object_2d_sw.cpp:148
    #9 0x112ea1a6 in Area2DSW::set_monitorable(bool) servers/physics_2d/area_2d_sw.cpp:197
    #10 0x1074d11b in Physics2DServerSW::area_set_monitorable(RID, bool) servers/physics_2d/physics_2d_server_sw.cpp:488
    #11 0x107c94b4 in Physics2DServerWrapMT::area_set_monitorable(RID, bool) servers/physics_2d/physics_2d_server_wrap_mt.h:165
    #12 0x100c8e60 in MethodBind2<RID, bool>::call(Object*, Variant const**, int, Variant::CallError&) core/method_bind.gen.inc:1523
    #13 0x11d2941f in Object::call(StringName const&, Variant const**, int, Variant::CallError&) core/object.cpp:918
    #14 0x11fb1898 in Variant::call_ptr(StringName const&, Variant const**, int, Variant*, Variant::CallError&) core/variant_call.cpp:1175
    #15 0x1e60f19 in GDScriptFunction::call(GDScriptInstance*, Variant const**, int, Variant::CallError&, GDScriptFunction::CallState*) modules/gdscript/gdscript_function.cpp:1046
    #16 0x1c91e7b in GDScriptInstance::call(StringName const&, Variant const**, int, Variant::CallError&) modules/gdscript/gdscript.cpp:1169
    #17 0x11d28f8f in Object::call(StringName const&, Variant const**, int, Variant::CallError&) core/object.cpp:899
    #18 0x11fb1898 in Variant::call_ptr(StringName const&, Variant const**, int, Variant*, Variant::CallError&) core/variant_call.cpp:1175
    #19 0x1e60f19 in GDScriptFunction::call(GDScriptInstance*, Variant const**, int, Variant::CallError&, GDScriptFunction::CallState*) modules/gdscript/gdscript_function.cpp:1046
    #20 0x1c91e7b in GDScriptInstance::call(StringName const&, Variant const**, int, Variant::CallError&) modules/gdscript/gdscript.cpp:1169
    #21 0x11d28f8f in Object::call(StringName const&, Variant const**, int, Variant::CallError&) core/object.cpp:899
    #22 0x11d33280 in Object::emit_signal(StringName const&, Variant const**, int) core/object.cpp:1224
    #23 0x11d352b6 in Object::emit_signal(StringName const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&) core/object.cpp:1279
    #24 0xc685a9b in Timer::_notification(int) scene/main/timer.cpp:77
    #25 0xc6906f7 in Timer::_notificationv(int, bool) scene/main/timer.h:37
    #26 0x11d298b9 in Object::notification(int, bool) core/object.cpp:927
    #27 0xc5e0ec9 in SceneTree::_notify_group_pause(StringName const&, int) scene/main/scene_tree.cpp:973
    #28 0xc5ce525 in SceneTree::iteration(float) scene/main/scene_tree.cpp:480
    #29 0x1981b4e in Main::iteration() main/main.cpp:2156
    #30 0x185ac8a in OS_X11::run() platform/x11/os_x11.cpp:3641
    #31 0x17c5d8b in main platform/x11/godot_x11.cpp:55
    #32 0x7fb5ce510564 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x28564)
    #33 0x17c59ad in _start (/usr/bin/godots+0x17c59ad)

Address 0x7ffdae73dff8 is located in stack of thread T0 at offset 40 in frame
    #0 0x113ad3e9 in BVH_Tree<CollisionObject2DSW, 2, 128, true, Rect2, Vector2>::_split_leaf_sort_groups_simple(int&, int&, unsigned short*, unsigned short*, BVH_ABB<Rect2, Vector2> const*, BVH_ABB<Rect2, Vector2>) core/math/bvh_split.inc:14

  This frame has 5 object(s):
    [32, 40) 'centre' (line 28) <== Memory access at offset 40 overflows this variable
    [64, 72) 'size' (line 29)
    [96, 108) 'order' (line 31)
    [128, 140) 'min_group_size' (line 57)
    [160, 176) 'full_bound' (line 14)
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow core/math/bvh_split.inc:74 in BVH_Tree<CollisionObject2DSW, 2, 128, true, Rect2, Vector2>::_split_leaf_sort_groups_simple(int&, int&, unsigned short*, unsigned short*, BVH_ABB<Rect2, Vector2> const*, BVH_ABB<Rect2, Vector2>)

Steps to reproduce

Run minimal project with asan/ubsan

Minimal reproduction project

BulletSpawnerTest.zip

[lawnjelly]

Ah looks like this is one for me and @pouleyKetchoupp . I originally wrote the BVH for 3D, but pouley modified it to work for 2D as well in #48314, and we must have missed this area which is assuming there are 3 axes.