Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Possible heap-use-after-free in ctypes in Python 3.12 #113576

Open
jiridanek opened this issue Dec 29, 2023 · 5 comments
Open

Possible heap-use-after-free in ctypes in Python 3.12 #113576

jiridanek opened this issue Dec 29, 2023 · 5 comments
Labels
3.12 bugs and security fixes topic-ctypes topic-subinterpreters type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@jiridanek
Copy link

jiridanek commented Dec 29, 2023 •
edited
Loading

Crash report

What happened?

I have a binary which is embedding python. I want to test this binary with address sanitizer, and leak sanitizer. Regular python3.12 build contained in Fedora 39 causes lsan to report leaks even with PYTHONMALLOC=malloc_debug, which with previous versions gave me clean run.

Therefore, I tried python3-debug, which is a debug build of python packaged in Fedora. With that, I hit

and I was playing around a bit. When I hit the mentioned issue, I was using PYTHONMALLOC="malloc_debug". I wanted to try other values for this variable. With pymalloc_debug, I got the same assertion.

With malloc, I got

=================================================================
==1716614==ERROR: AddressSanitizer: heap-use-after-free on address 0x6040000aba78 at pc 0x7f2ba546eb32 bp 0x7f2b9e24e520 sp 0x7f2b9e24dce0
READ of size 3 at 0x6040000aba78 thread T4
    #0 0x7f2ba546eb31 in __interceptor_memcpy (/lib64/libasan.so.8+0x6eb31) (BuildId: 7fcb7759bc17ef47f9682414b6d99732d6a6ab0c)
    #1 0x7f2ba4cd6d8a in _copy_characters (/lib64/libpython3.12d.so.1.0+0x2d6d8a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #2 0x7f2ba4cd7831 in _PyUnicode_FastCopyCharacters (/lib64/libpython3.12d.so.1.0+0x2d7831) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #3 0x7f2ba4cf54ef in PyUnicode_Concat (/lib64/libpython3.12d.so.1.0+0x2f54ef) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #4 0x7f2ba0a44b51 in CreateSwappedType (/usr/lib64/python3.12/lib-dynload/_ctypes.cpython-312d-x86_64-linux-gnu.so+0xcb51) (BuildId: cdb5a61ddd026358017a72a3d3405dd431e31b72)
    #5 0x7f2ba0a4552c in PyCSimpleType_new (/usr/lib64/python3.12/lib-dynload/_ctypes.cpython-312d-x86_64-linux-gnu.so+0xd52c) (BuildId: cdb5a61ddd026358017a72a3d3405dd431e31b72)
    #6 0x7f2ba4c958c3 in type_call (/lib64/libpython3.12d.so.1.0+0x2958c3) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #7 0x7f2ba4bf8792 in _PyObject_MakeTpCall (/lib64/libpython3.12d.so.1.0+0x1f8792) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #8 0x7f2ba4bf825c in _PyObject_FastCallDictTstate (/lib64/libpython3.12d.so.1.0+0x1f825c) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #9 0x7f2ba4bf8389 in PyObject_VectorcallDict (/lib64/libpython3.12d.so.1.0+0x1f8389) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #10 0x7f2ba4d5afd1 in builtin___build_class__ (/lib64/libpython3.12d.so.1.0+0x35afd1) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #11 0x7f2ba4c6df9a in cfunction_vectorcall_FASTCALL_KEYWORDS (/lib64/libpython3.12d.so.1.0+0x26df9a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #12 0x7f2ba4bf7c81 in _PyObject_VectorcallTstate.lto_priv.3 (/lib64/libpython3.12d.so.1.0+0x1f7c81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #13 0x7f2ba4bf8b98 in PyObject_Vectorcall (/lib64/libpython3.12d.so.1.0+0x1f8b98) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #14 0x7f2ba4d7b662 in _PyEval_EvalFrameDefault (/lib64/libpython3.12d.so.1.0+0x37b662) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #15 0x7f2ba4d61f81 in _PyEval_EvalFrame.lto_priv.1 (/lib64/libpython3.12d.so.1.0+0x361f81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #16 0x7f2ba4d86a6c in _PyEval_Vector (/lib64/libpython3.12d.so.1.0+0x386a6c) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #17 0x7f2ba4d63933 in PyEval_EvalCode (/lib64/libpython3.12d.so.1.0+0x363933) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #18 0x7f2ba4d5c678 in builtin_exec_impl (/lib64/libpython3.12d.so.1.0+0x35c678) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #19 0x7f2ba4d59a3a in builtin_exec (/lib64/libpython3.12d.so.1.0+0x359a3a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #20 0x7f2ba4c6df9a in cfunction_vectorcall_FASTCALL_KEYWORDS (/lib64/libpython3.12d.so.1.0+0x26df9a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #21 0x7f2ba4bf8909 in _PyVectorcall_Call (/lib64/libpython3.12d.so.1.0+0x1f8909) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #22 0x7f2ba4bf8cf1 in _PyObject_Call (/lib64/libpython3.12d.so.1.0+0x1f8cf1) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #23 0x7f2ba4bf8dcc in PyObject_Call (/lib64/libpython3.12d.so.1.0+0x1f8dcc) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #24 0x7f2ba4d814dc in _PyEval_EvalFrameDefault (/lib64/libpython3.12d.so.1.0+0x3814dc) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #25 0x7f2ba4d61f81 in _PyEval_EvalFrame.lto_priv.1 (/lib64/libpython3.12d.so.1.0+0x361f81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #26 0x7f2ba4d86a6c in _PyEval_Vector (/lib64/libpython3.12d.so.1.0+0x386a6c) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #27 0x7f2ba4bf8fdd in _PyFunction_Vectorcall (/lib64/libpython3.12d.so.1.0+0x1f8fdd) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #28 0x7f2ba4bf7c81 in _PyObject_VectorcallTstate.lto_priv.3 (/lib64/libpython3.12d.so.1.0+0x1f7c81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #29 0x7f2ba4bfa545 in object_vacall (/lib64/libpython3.12d.so.1.0+0x1fa545) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #30 0x7f2ba4bfa84a in PyObject_CallMethodObjArgs (/lib64/libpython3.12d.so.1.0+0x1fa84a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #31 0x7f2ba4ddba6b in import_find_and_load (/lib64/libpython3.12d.so.1.0+0x3dba6b) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #32 0x7f2ba4ddbdb8 in PyImport_ImportModuleLevelObject (/lib64/libpython3.12d.so.1.0+0x3dbdb8) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #33 0x7f2ba4d88907 in import_name (/lib64/libpython3.12d.so.1.0+0x388907) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #34 0x7f2ba4d76e96 in _PyEval_EvalFrameDefault (/lib64/libpython3.12d.so.1.0+0x376e96) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #35 0x7f2ba4d61f81 in _PyEval_EvalFrame.lto_priv.1 (/lib64/libpython3.12d.so.1.0+0x361f81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #36 0x7f2ba4d86a6c in _PyEval_Vector (/lib64/libpython3.12d.so.1.0+0x386a6c) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #37 0x7f2ba4d63933 in PyEval_EvalCode (/lib64/libpython3.12d.so.1.0+0x363933) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #38 0x7f2ba4d5c678 in builtin_exec_impl (/lib64/libpython3.12d.so.1.0+0x35c678) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #39 0x7f2ba4d59a3a in builtin_exec (/lib64/libpython3.12d.so.1.0+0x359a3a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #40 0x7f2ba4c6df9a in cfunction_vectorcall_FASTCALL_KEYWORDS (/lib64/libpython3.12d.so.1.0+0x26df9a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #41 0x7f2ba4bf8909 in _PyVectorcall_Call (/lib64/libpython3.12d.so.1.0+0x1f8909) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #42 0x7f2ba4bf8cf1 in _PyObject_Call (/lib64/libpython3.12d.so.1.0+0x1f8cf1) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #43 0x7f2ba4bf8dcc in PyObject_Call (/lib64/libpython3.12d.so.1.0+0x1f8dcc) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #44 0x7f2ba4d814dc in _PyEval_EvalFrameDefault (/lib64/libpython3.12d.so.1.0+0x3814dc) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #45 0x7f2ba4d61f81 in _PyEval_EvalFrame.lto_priv.1 (/lib64/libpython3.12d.so.1.0+0x361f81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #46 0x7f2ba4d86a6c in _PyEval_Vector (/lib64/libpython3.12d.so.1.0+0x386a6c) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #47 0x7f2ba4bf8fdd in _PyFunction_Vectorcall (/lib64/libpython3.12d.so.1.0+0x1f8fdd) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #48 0x7f2ba4bf7c81 in _PyObject_VectorcallTstate.lto_priv.3 (/lib64/libpython3.12d.so.1.0+0x1f7c81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #49 0x7f2ba4bfa545 in object_vacall (/lib64/libpython3.12d.so.1.0+0x1fa545) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #50 0x7f2ba4bfa84a in PyObject_CallMethodObjArgs (/lib64/libpython3.12d.so.1.0+0x1fa84a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #51 0x7f2ba4ddba6b in import_find_and_load (/lib64/libpython3.12d.so.1.0+0x3dba6b) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #52 0x7f2ba4ddbdb8 in PyImport_ImportModuleLevelObject (/lib64/libpython3.12d.so.1.0+0x3dbdb8) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #53 0x7f2ba4d5b1ee in builtin___import___impl (/lib64/libpython3.12d.so.1.0+0x35b1ee) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #54 0x7f2ba4d591d8 in builtin___import__ (/lib64/libpython3.12d.so.1.0+0x3591d8) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #55 0x7f2ba4c6df9a in cfunction_vectorcall_FASTCALL_KEYWORDS (/lib64/libpython3.12d.so.1.0+0x26df9a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #56 0x7f2ba4bf7c81 in _PyObject_VectorcallTstate.lto_priv.3 (/lib64/libpython3.12d.so.1.0+0x1f7c81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #57 0x7f2ba4bf9595 in _PyObject_CallFunctionVa (/lib64/libpython3.12d.so.1.0+0x1f9595) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #58 0x7f2ba4bf96e5 in PyObject_CallFunction (/lib64/libpython3.12d.so.1.0+0x1f96e5) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #59 0x7f2ba4ddc3f5 in PyImport_Import (/lib64/libpython3.12d.so.1.0+0x3dc3f5) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #60 0x7f2ba4ddae85 in PyImport_ImportModule (/lib64/libpython3.12d.so.1.0+0x3dae85) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #61 0x5e5700 in qd_python_setup /home/jdanek/repos/skupper-router/src/python_embedded.c:841
    #62 0x57b5a0 in qd_python_initialize /home/jdanek/repos/skupper-router/src/python_embedded.c:58
    #63 0x57b5a0 in qd_dispatch /home/jdanek/repos/skupper-router/src/dispatch.c:137
    #64 0x7b19ef in QDR::initialize(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /home/jdanek/repos/skupper-router/tests/c_unittests/helpers.hpp:206
    #65 0x8974bc in operator() /home/jdanek/repos/skupper-router/tests/c_unittests/test_http_listener_connector.cpp:67
    #66 0x7f2ba3ee31e2 in execute_native_thread_routine (/lib64/libstdc++.so.6+0xe31e2) (BuildId: 43b5fe9c4fe0185772be4b57539be137c14a5c05)
    #67 0x7f2ba3cac896 in start_thread (/lib64/libc.so.6+0x8e896) (BuildId: 788cdd41a15985bf8e0a48d213a46e07d58822df)
    #68 0x7f2ba3d336fb in clone3 (/lib64/libc.so.6+0x1156fb) (BuildId: 788cdd41a15985bf8e0a48d213a46e07d58822df)

0x6040000aba78 is located 40 bytes inside of 44-byte region [0x6040000aba50,0x6040000aba7c)
freed by thread T1 here:
    #0 0x7f2ba54d7fb8 in __interceptor_free.part.0 (/lib64/libasan.so.8+0xd7fb8) (BuildId: 7fcb7759bc17ef47f9682414b6d99732d6a6ab0c)
    #1 0x7f2ba4c77b26 in _PyMem_RawFree (/lib64/libpython3.12d.so.1.0+0x277b26) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #2 0x7f2ba4c792a2 in PyObject_Free (/lib64/libpython3.12d.so.1.0+0x2792a2) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #3 0x7f2ba4cd7dcf in unicode_dealloc (/lib64/libpython3.12d.so.1.0+0x2d7dcf) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #4 0x7f2ba4c7778a in _Py_Dealloc (/lib64/libpython3.12d.so.1.0+0x27778a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #5 0x7f2ba4c50b77 in Py_DECREF.lto_priv.27 (/lib64/libpython3.12d.so.1.0+0x250b77) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #6 0x7f2ba4c50bcc in Py_XDECREF.lto_priv.25 (/lib64/libpython3.12d.so.1.0+0x250bcc) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #7 0x7f2ba4c533c8 in free_keys_object (/lib64/libpython3.12d.so.1.0+0x2533c8) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #8 0x7f2ba4c524ca in dictkeys_decref (/lib64/libpython3.12d.so.1.0+0x2524ca) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #9 0x7f2ba4c575ba in PyDict_Clear (/lib64/libpython3.12d.so.1.0+0x2575ba) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #10 0x7f2ba4cc4c93 in clear_interned_dict (/lib64/libpython3.12d.so.1.0+0x2c4c93) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #11 0x7f2ba4d0136c in _PyUnicode_ClearInterned (/lib64/libpython3.12d.so.1.0+0x30136c) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #12 0x7f2ba4e0da25 in finalize_interp_types (/lib64/libpython3.12d.so.1.0+0x40da25) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #13 0x7f2ba4e0db0c in finalize_interp_clear (/lib64/libpython3.12d.so.1.0+0x40db0c) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #14 0x7f2ba4e0dc99 in Py_FinalizeEx (/lib64/libpython3.12d.so.1.0+0x40dc99) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #15 0x7f2ba4e0dd07 in Py_Finalize (/lib64/libpython3.12d.so.1.0+0x40dd07) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #16 0x5ccdb0 in qd_python_finalize /home/jdanek/repos/skupper-router/src/python_embedded.c:71
    #17 0x57ac8a in qd_dispatch_free /home/jdanek/repos/skupper-router/src/dispatch.c:394
    #18 0x879f06 in qd_dispatch_free /home/jdanek/repos/skupper-router/src/dispatch.c:374
    #19 0x879f06 in QDR::deinitialize(bool) const [clone .constprop.0] /home/jdanek/repos/skupper-router/tests/c_unittests/helpers.hpp:264
    #20 0x8a95be in operator() /home/jdanek/repos/skupper-router/tests/c_unittests/test_connection_manager_static.cpp:131
    #21 0x7f2ba3ee31e2 in execute_native_thread_routine (/lib64/libstdc++.so.6+0xe31e2) (BuildId: 43b5fe9c4fe0185772be4b57539be137c14a5c05)

previously allocated by thread T1 here:
    #0 0x7f2ba54d92ef in malloc (/lib64/libasan.so.8+0xd92ef) (BuildId: 7fcb7759bc17ef47f9682414b6d99732d6a6ab0c)
    #1 0x7f2ba4c77a7d in _PyMem_RawMalloc (/lib64/libpython3.12d.so.1.0+0x277a7d) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #2 0x7f2ba4c791ab in PyObject_Malloc (/lib64/libpython3.12d.so.1.0+0x2791ab) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #3 0x7f2ba4cd6550 in PyUnicode_New (/lib64/libpython3.12d.so.1.0+0x2d6550) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #4 0x7f2ba4ce4c72 in unicode_decode_utf8 (/lib64/libpython3.12d.so.1.0+0x2e4c72) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #5 0x7f2ba4ce523f in PyUnicode_DecodeUTF8Stateful (/lib64/libpython3.12d.so.1.0+0x2e523f) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #6 0x7f2ba4cd8da1 in PyUnicode_FromString (/lib64/libpython3.12d.so.1.0+0x2d8da1) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #7 0x7f2ba4d01125 in PyUnicode_InternFromString (/lib64/libpython3.12d.so.1.0+0x301125) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #8 0x7f2ba0a44b03 in CreateSwappedType (/usr/lib64/python3.12/lib-dynload/_ctypes.cpython-312d-x86_64-linux-gnu.so+0xcb03) (BuildId: cdb5a61ddd026358017a72a3d3405dd431e31b72)
    #9 0x7f2ba0a4552c in PyCSimpleType_new (/usr/lib64/python3.12/lib-dynload/_ctypes.cpython-312d-x86_64-linux-gnu.so+0xd52c) (BuildId: cdb5a61ddd026358017a72a3d3405dd431e31b72)
    #10 0x7f2ba4c958c3 in type_call (/lib64/libpython3.12d.so.1.0+0x2958c3) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #11 0x7f2ba4bf8792 in _PyObject_MakeTpCall (/lib64/libpython3.12d.so.1.0+0x1f8792) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #12 0x7f2ba4bf825c in _PyObject_FastCallDictTstate (/lib64/libpython3.12d.so.1.0+0x1f825c) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #13 0x7f2ba4bf8389 in PyObject_VectorcallDict (/lib64/libpython3.12d.so.1.0+0x1f8389) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #14 0x7f2ba4d5afd1 in builtin___build_class__ (/lib64/libpython3.12d.so.1.0+0x35afd1) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #15 0x7f2ba4c6df9a in cfunction_vectorcall_FASTCALL_KEYWORDS (/lib64/libpython3.12d.so.1.0+0x26df9a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #16 0x7f2ba4bf7c81 in _PyObject_VectorcallTstate.lto_priv.3 (/lib64/libpython3.12d.so.1.0+0x1f7c81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #17 0x7f2ba4bf8b98 in PyObject_Vectorcall (/lib64/libpython3.12d.so.1.0+0x1f8b98) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #18 0x7f2ba4d7b662 in _PyEval_EvalFrameDefault (/lib64/libpython3.12d.so.1.0+0x37b662) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #19 0x7f2ba4d61f81 in _PyEval_EvalFrame.lto_priv.1 (/lib64/libpython3.12d.so.1.0+0x361f81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #20 0x7f2ba4d86a6c in _PyEval_Vector (/lib64/libpython3.12d.so.1.0+0x386a6c) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #21 0x7f2ba4d63933 in PyEval_EvalCode (/lib64/libpython3.12d.so.1.0+0x363933) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #22 0x7f2ba4d5c678 in builtin_exec_impl (/lib64/libpython3.12d.so.1.0+0x35c678) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #23 0x7f2ba4d59a3a in builtin_exec (/lib64/libpython3.12d.so.1.0+0x359a3a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #24 0x7f2ba4c6df9a in cfunction_vectorcall_FASTCALL_KEYWORDS (/lib64/libpython3.12d.so.1.0+0x26df9a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #25 0x7f2ba4bf8909 in _PyVectorcall_Call (/lib64/libpython3.12d.so.1.0+0x1f8909) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #26 0x7f2ba4bf8cf1 in _PyObject_Call (/lib64/libpython3.12d.so.1.0+0x1f8cf1) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #27 0x7f2ba4bf8dcc in PyObject_Call (/lib64/libpython3.12d.so.1.0+0x1f8dcc) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #28 0x7f2ba4d814dc in _PyEval_EvalFrameDefault (/lib64/libpython3.12d.so.1.0+0x3814dc) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #29 0x7f2ba4d61f81 in _PyEval_EvalFrame.lto_priv.1 (/lib64/libpython3.12d.so.1.0+0x361f81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)

Thread T4 created by T0 here:
    #0 0x7f2ba5448956 in pthread_create (/lib64/libasan.so.8+0x48956) (BuildId: 7fcb7759bc17ef47f9682414b6d99732d6a6ab0c)
    #1 0x7f2ba3ee32b8 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/lib64/libstdc++.so.6+0xe32b8) (BuildId: 43b5fe9c4fe0185772be4b57539be137c14a5c05)
    #2 0x7ca08e in DOCTEST_ANON_FUNC_29 /home/jdanek/repos/skupper-router/tests/c_unittests/test_http_listener_connector.cpp:65
    #3 0x78a350 in doctest::Context::run() /home/jdanek/repos/skupper-router/tests/c_unittests/doctest.h:7007
    #4 0x456aad in main /home/jdanek/repos/skupper-router/tests/c_unittests/c_unittests_main.cpp:76
    #5 0x7f2ba3c46149 in __libc_start_call_main (/lib64/libc.so.6+0x28149) (BuildId: 788cdd41a15985bf8e0a48d213a46e07d58822df)
    #6 0x7f2ba3c4620a in __libc_start_main_impl (/lib64/libc.so.6+0x2820a) (BuildId: 788cdd41a15985bf8e0a48d213a46e07d58822df)
    #7 0x45c874 in _start (/home/jdanek/repos/skupper-router/cmake-build-relwithdebinfo-asan/tests/c_unittests/c_unittests+0x45c874) (BuildId: 7638f1da81a091da1a78ee3ba7675e8ad027ba1d)

Thread T1 created by T0 here:
    #0 0x7f2ba5448956 in pthread_create (/lib64/libasan.so.8+0x48956) (BuildId: 7fcb7759bc17ef47f9682414b6d99732d6a6ab0c)
    #1 0x7f2ba3ee32b8 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/lib64/libstdc++.so.6+0xe32b8) (BuildId: 43b5fe9c4fe0185772be4b57539be137c14a5c05)
    #2 0x7c9dde in DOCTEST_ANON_FUNC_26 /home/jdanek/repos/skupper-router/tests/c_unittests/test_connection_manager_static.cpp:71
    #3 0x78a350 in doctest::Context::run() /home/jdanek/repos/skupper-router/tests/c_unittests/doctest.h:7007
    #4 0x456aad in main /home/jdanek/repos/skupper-router/tests/c_unittests/c_unittests_main.cpp:76
    #5 0x7f2ba3c46149 in __libc_start_call_main (/lib64/libc.so.6+0x28149) (BuildId: 788cdd41a15985bf8e0a48d213a46e07d58822df)
    #6 0x7f2ba3c4620a in __libc_start_main_impl (/lib64/libc.so.6+0x2820a) (BuildId: 788cdd41a15985bf8e0a48d213a46e07d58822df)
    #7 0x45c874 in _start (/home/jdanek/repos/skupper-router/cmake-build-relwithdebinfo-asan/tests/c_unittests/c_unittests+0x45c874) (BuildId: 7638f1da81a091da1a78ee3ba7675e8ad027ba1d)

SUMMARY: AddressSanitizer: heap-use-after-free (/lib64/libasan.so.8+0x6eb31) (BuildId: 7fcb7759bc17ef47f9682414b6d99732d6a6ab0c) in __interceptor_memcpy
Shadow bytes around the buggy address:
  0x6040000ab780: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fd
  0x6040000ab800: fa fa 00 00 00 00 00 00 fa fa fd fd fd fd fd fd
  0x6040000ab880: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 00
  0x6040000ab900: fa fa 00 00 00 00 00 07 fa fa 00 00 00 00 00 00
  0x6040000ab980: fa fa 00 00 00 00 00 06 fa fa fd fd fd fd fd fd
=>0x6040000aba00: fa fa 00 00 00 00 00 fa fa fa fd fd fd fd fd[fd]
  0x6040000aba80: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fd
  0x6040000abb00: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 00
  0x6040000abb80: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 00
  0x6040000abc00: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 00
  0x6040000abc80: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==1716614==ABORTING

Process finished with exit code 1

and with pymalloc I got a different trace

=================================================================
==1717283==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x7f483bee2310 in thread T4
    #0 0x7f48420d7fb8 in __interceptor_free.part.0 (/lib64/libasan.so.8+0xd7fb8) (BuildId: 7fcb7759bc17ef47f9682414b6d99732d6a6ab0c)
    #1 0x7f4841877b26 in _PyMem_RawFree (/lib64/libpython3.12d.so.1.0+0x277b26) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #2 0x7f4841878e51 in PyMem_RawFree (/lib64/libpython3.12d.so.1.0+0x278e51) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #3 0x7f484187b787 in _PyObject_Free (/lib64/libpython3.12d.so.1.0+0x27b787) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #4 0x7f48418792a2 in PyObject_Free (/lib64/libpython3.12d.so.1.0+0x2792a2) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #5 0x7f4841855e50 in dictresize (/lib64/libpython3.12d.so.1.0+0x255e50) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #6 0x7f4841854949 in insertion_resize (/lib64/libpython3.12d.so.1.0+0x254949) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #7 0x7f4841854ce6 in insertdict (/lib64/libpython3.12d.so.1.0+0x254ce6) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #8 0x7f48418569c4 in _PyDict_SetItem_Take2 (/lib64/libpython3.12d.so.1.0+0x2569c4) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #9 0x7f4841856aa8 in PyDict_SetItem (/lib64/libpython3.12d.so.1.0+0x256aa8) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #10 0x7f48418a4bf3 in add_subclass (/lib64/libpython3.12d.so.1.0+0x2a4bf3) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #11 0x7f48418a41a4 in type_ready_add_subclasses (/lib64/libpython3.12d.so.1.0+0x2a41a4) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #12 0x7f48418a45f4 in type_ready (/lib64/libpython3.12d.so.1.0+0x2a45f4) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #13 0x7f48418a47ad in PyType_Ready (/lib64/libpython3.12d.so.1.0+0x2a47ad) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #14 0x7f484189a983 in type_new_impl (/lib64/libpython3.12d.so.1.0+0x29a983) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #15 0x7f484189af7b in type_new (/lib64/libpython3.12d.so.1.0+0x29af7b) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #16 0x7f483cbe0f52 in PyCFuncPtrType_new (/usr/lib64/python3.12/lib-dynload/_ctypes.cpython-312d-x86_64-linux-gnu.so+0xdf52) (BuildId: cdb5a61ddd026358017a72a3d3405dd431e31b72)
    #17 0x7f48418958c3 in type_call (/lib64/libpython3.12d.so.1.0+0x2958c3) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #18 0x7f48417f8792 in _PyObject_MakeTpCall (/lib64/libpython3.12d.so.1.0+0x1f8792) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #19 0x7f48417f825c in _PyObject_FastCallDictTstate (/lib64/libpython3.12d.so.1.0+0x1f825c) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #20 0x7f48417f8389 in PyObject_VectorcallDict (/lib64/libpython3.12d.so.1.0+0x1f8389) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #21 0x7f484195afd1 in builtin___build_class__ (/lib64/libpython3.12d.so.1.0+0x35afd1) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #22 0x7f484197e62b in _PyEval_EvalFrameDefault (/lib64/libpython3.12d.so.1.0+0x37e62b) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #23 0x7f4841961f81 in _PyEval_EvalFrame.lto_priv.1 (/lib64/libpython3.12d.so.1.0+0x361f81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #24 0x7f4841986a6c in _PyEval_Vector (/lib64/libpython3.12d.so.1.0+0x386a6c) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #25 0x7f4841963933 in PyEval_EvalCode (/lib64/libpython3.12d.so.1.0+0x363933) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #26 0x7f484195c678 in builtin_exec_impl (/lib64/libpython3.12d.so.1.0+0x35c678) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #27 0x7f4841959a3a in builtin_exec (/lib64/libpython3.12d.so.1.0+0x359a3a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #28 0x7f484186df9a in cfunction_vectorcall_FASTCALL_KEYWORDS (/lib64/libpython3.12d.so.1.0+0x26df9a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #29 0x7f48417f8909 in _PyVectorcall_Call (/lib64/libpython3.12d.so.1.0+0x1f8909) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #30 0x7f48417f8cf1 in _PyObject_Call (/lib64/libpython3.12d.so.1.0+0x1f8cf1) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #31 0x7f48417f8dcc in PyObject_Call (/lib64/libpython3.12d.so.1.0+0x1f8dcc) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #32 0x7f48419814dc in _PyEval_EvalFrameDefault (/lib64/libpython3.12d.so.1.0+0x3814dc) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #33 0x7f4841961f81 in _PyEval_EvalFrame.lto_priv.1 (/lib64/libpython3.12d.so.1.0+0x361f81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #34 0x7f4841986a6c in _PyEval_Vector (/lib64/libpython3.12d.so.1.0+0x386a6c) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #35 0x7f48417f8fdd in _PyFunction_Vectorcall (/lib64/libpython3.12d.so.1.0+0x1f8fdd) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #36 0x7f48417f7c81 in _PyObject_VectorcallTstate.lto_priv.3 (/lib64/libpython3.12d.so.1.0+0x1f7c81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #37 0x7f48417fa545 in object_vacall (/lib64/libpython3.12d.so.1.0+0x1fa545) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #38 0x7f48417fa84a in PyObject_CallMethodObjArgs (/lib64/libpython3.12d.so.1.0+0x1fa84a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #39 0x7f48419dba6b in import_find_and_load (/lib64/libpython3.12d.so.1.0+0x3dba6b) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #40 0x7f48419dbdb8 in PyImport_ImportModuleLevelObject (/lib64/libpython3.12d.so.1.0+0x3dbdb8) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #41 0x7f4841988907 in import_name (/lib64/libpython3.12d.so.1.0+0x388907) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #42 0x7f4841976e96 in _PyEval_EvalFrameDefault (/lib64/libpython3.12d.so.1.0+0x376e96) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #43 0x7f4841961f81 in _PyEval_EvalFrame.lto_priv.1 (/lib64/libpython3.12d.so.1.0+0x361f81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #44 0x7f4841986a6c in _PyEval_Vector (/lib64/libpython3.12d.so.1.0+0x386a6c) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #45 0x7f4841963933 in PyEval_EvalCode (/lib64/libpython3.12d.so.1.0+0x363933) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #46 0x7f484195c678 in builtin_exec_impl (/lib64/libpython3.12d.so.1.0+0x35c678) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #47 0x7f4841959a3a in builtin_exec (/lib64/libpython3.12d.so.1.0+0x359a3a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #48 0x7f484186df9a in cfunction_vectorcall_FASTCALL_KEYWORDS (/lib64/libpython3.12d.so.1.0+0x26df9a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #49 0x7f48417f8909 in _PyVectorcall_Call (/lib64/libpython3.12d.so.1.0+0x1f8909) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #50 0x7f48417f8cf1 in _PyObject_Call (/lib64/libpython3.12d.so.1.0+0x1f8cf1) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #51 0x7f48417f8dcc in PyObject_Call (/lib64/libpython3.12d.so.1.0+0x1f8dcc) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #52 0x7f48419814dc in _PyEval_EvalFrameDefault (/lib64/libpython3.12d.so.1.0+0x3814dc) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #53 0x7f4841961f81 in _PyEval_EvalFrame.lto_priv.1 (/lib64/libpython3.12d.so.1.0+0x361f81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #54 0x7f4841986a6c in _PyEval_Vector (/lib64/libpython3.12d.so.1.0+0x386a6c) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #55 0x7f48417f8fdd in _PyFunction_Vectorcall (/lib64/libpython3.12d.so.1.0+0x1f8fdd) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #56 0x7f48417f7c81 in _PyObject_VectorcallTstate.lto_priv.3 (/lib64/libpython3.12d.so.1.0+0x1f7c81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #57 0x7f48417fa545 in object_vacall (/lib64/libpython3.12d.so.1.0+0x1fa545) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #58 0x7f48417fa84a in PyObject_CallMethodObjArgs (/lib64/libpython3.12d.so.1.0+0x1fa84a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #59 0x7f48419dba6b in import_find_and_load (/lib64/libpython3.12d.so.1.0+0x3dba6b) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #60 0x7f48419dbdb8 in PyImport_ImportModuleLevelObject (/lib64/libpython3.12d.so.1.0+0x3dbdb8) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #61 0x7f484195b1ee in builtin___import___impl (/lib64/libpython3.12d.so.1.0+0x35b1ee) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #62 0x7f48419591d8 in builtin___import__ (/lib64/libpython3.12d.so.1.0+0x3591d8) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #63 0x7f484186df9a in cfunction_vectorcall_FASTCALL_KEYWORDS (/lib64/libpython3.12d.so.1.0+0x26df9a) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #64 0x7f48417f7c81 in _PyObject_VectorcallTstate.lto_priv.3 (/lib64/libpython3.12d.so.1.0+0x1f7c81) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #65 0x7f48417f9595 in _PyObject_CallFunctionVa (/lib64/libpython3.12d.so.1.0+0x1f9595) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #66 0x7f48417f96e5 in PyObject_CallFunction (/lib64/libpython3.12d.so.1.0+0x1f96e5) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #67 0x7f48419dc3f5 in PyImport_Import (/lib64/libpython3.12d.so.1.0+0x3dc3f5) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #68 0x7f48419dae85 in PyImport_ImportModule (/lib64/libpython3.12d.so.1.0+0x3dae85) (BuildId: 3c61128b1fd63fa1904863ae8c5c9718b1cd55c4)
    #69 0x5e5700 in qd_python_setup /home/jdanek/repos/skupper-router/src/python_embedded.c:841
    #70 0x57b5a0 in qd_python_initialize /home/jdanek/repos/skupper-router/src/python_embedded.c:58
    #71 0x57b5a0 in qd_dispatch /home/jdanek/repos/skupper-router/src/dispatch.c:137
    #72 0x7b19ef in QDR::initialize(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /home/jdanek/repos/skupper-router/tests/c_unittests/helpers.hpp:206
    #73 0x8974bc in operator() /home/jdanek/repos/skupper-router/tests/c_unittests/test_http_listener_connector.cpp:67
    #74 0x7f4840ae31e2 in execute_native_thread_routine (/lib64/libstdc++.so.6+0xe31e2) (BuildId: 43b5fe9c4fe0185772be4b57539be137c14a5c05)
    #75 0x7f48408ac896 in start_thread (/lib64/libc.so.6+0x8e896) (BuildId: 788cdd41a15985bf8e0a48d213a46e07d58822df)
    #76 0x7f48409336fb in clone3 (/lib64/libc.so.6+0x1156fb) (BuildId: 788cdd41a15985bf8e0a48d213a46e07d58822df)

Address 0x7f483bee2310 is a wild pointer inside of access range of size 0x000000000001.
SUMMARY: AddressSanitizer: bad-free (/lib64/libasan.so.8+0xd7fb8) (BuildId: 7fcb7759bc17ef47f9682414b6d99732d6a6ab0c) in __interceptor_free.part.0
Thread T4 created by T0 here:
    #0 0x7f4842048956 in pthread_create (/lib64/libasan.so.8+0x48956) (BuildId: 7fcb7759bc17ef47f9682414b6d99732d6a6ab0c)
    #1 0x7f4840ae32b8 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/lib64/libstdc++.so.6+0xe32b8) (BuildId: 43b5fe9c4fe0185772be4b57539be137c14a5c05)
    #2 0x7ca08e in DOCTEST_ANON_FUNC_29 /home/jdanek/repos/skupper-router/tests/c_unittests/test_http_listener_connector.cpp:65
    #3 0x78a350 in doctest::Context::run() /home/jdanek/repos/skupper-router/tests/c_unittests/doctest.h:7007
    #4 0x456aad in main /home/jdanek/repos/skupper-router/tests/c_unittests/c_unittests_main.cpp:76
    #5 0x7f4840846149 in __libc_start_call_main (/lib64/libc.so.6+0x28149) (BuildId: 788cdd41a15985bf8e0a48d213a46e07d58822df)
    #6 0x7f484084620a in __libc_start_main_impl (/lib64/libc.so.6+0x2820a) (BuildId: 788cdd41a15985bf8e0a48d213a46e07d58822df)
    #7 0x45c874 in _start (/home/jdanek/repos/skupper-router/cmake-build-relwithdebinfo-asan/tests/c_unittests/c_unittests+0x45c874) (BuildId: 7638f1da81a091da1a78ee3ba7675e8ad027ba1d)

==1717283==ABORTING

Process finished with exit code 1

Does this look like a Python bug? I'd be happy to attempt a reproducer if the backtraces are deemed sufficiently interesting and worth investigating.

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

$ python3-debug -VV
Python 3.12.0 (main, Oct  2 2023, 00:00:00) [GCC 13.2.1 20230918 (Red Hat 13.2.1-3)]

Tasks
Beta Give feedback
No tasks being tracked yet.
@jiridanek jiridanek added the type-crash A hard crash of the interpreter, possibly with a core dump label Dec 29, 2023
@jiridanek jiridanek mentioned this issue Dec 29, 2023
Open
@jiridanek
Copy link
Author

The reproducer is actually quite easy. It is enough to initialize and deinitialize Python interpreter twice in the same process.

#include "./qdr_doctest.hpp"
#include "./helpers.hpp"  // must come after ./qdr_doctest.hpp

#include <Python.h>

#include <thread>

TEST_CASE("Initialize and deinitialize Python twice")
{
    std::thread([]() {
        Py_Initialize();
        PyObject *m = PyImport_ImportModule("ctypes");
        Py_DECREF(m);
        PyGC_Collect();
        Py_Finalize();
    }).join();

    std::thread([]() {
        Py_Initialize();
        PyObject *m = PyImport_ImportModule("ctypes");
        Py_DECREF(m);
        PyGC_Collect();
        Py_Finalize();
    }).join();
}

I arrived at this by simplifying my original test at https://github.com/skupperproject/skupper-router/blob/cfc8b4c6892dafc5dd0c86682b4db87a488f2451/tests/c_unittests/test_router_startup.cpp#L25

@jiridanek jiridanek changed the title heap-use-after-free and attempting free on address which was not malloc()-ed warnings from AddressSanitizer in Python 3.12 -debug heap-use-after-free and attempting free on address which was not malloc()-ed warnings from AddressSanitizer in embedded Python 3.12 -debug Dec 29, 2023
@sobolevn
Copy link
Member

cc @pablogsal

@pablogsal
Copy link
Member

This looks like an issue in ctypes so we need to look at what changed from 3.11 to 3.12. @jiridanek can you try to bisect using your reproducer?

@pablogsal pablogsal changed the title heap-use-after-free and attempting free on address which was not malloc()-ed warnings from AddressSanitizer in embedded Python 3.12 -debug Possible heap-use-after-free in ctypes in Python 3.12 Dec 31, 2023
@pablogsal pablogsal added the 3.12 bugs and security fixes label Dec 31, 2023
@jiridanek
Copy link
Author

jiridanek commented Jan 2, 2024 •
edited
Loading

@sobolevn @pablogsal here's a suspected culprit from git-bisect

67807cfc87135fdce4992d38d2ffe3e44747e73b is the first bad commit
commit 67807cfc87135fdce4992d38d2ffe3e44747e73b
Author: Eric Snow <ericsnowcurrently@gmail.com>
Date:   Fri Nov 11 16:30:46 2022 -0700

    gh-81057: Move the Allocators to _PyRuntimeState (gh-99217)
    
    The global allocators were stored in 3 static global variables: _PyMem_Raw, _PyMem, and _PyObject.  State for the "small block" allocator was stored in another 13.  That makes a total of 16 global variables. We are moving all 16 to the _PyRuntimeState struct as part of the work for gh-81057.  (If PEP 684 is accepted then we will follow up by moving them all to PyInterpreterState.)
    
    https://github.com/python/cpython/issues/81057

 Include/internal/pycore_obmalloc.h                 |  689 +++++++++++++
 Include/internal/pycore_obmalloc_init.h            |   68 ++
 Include/internal/pycore_pymem.h                    |   35 +-
 Include/internal/pycore_pymem_init.h               |   85 ++
 Include/internal/pycore_runtime.h                  |    5 +
 Include/internal/pycore_runtime_init.h             |   10 +-
 Makefile.pre.in                                    |    3 +
 .../2022-11-07-14-16-59.gh-issue-81057.3uKlLQ.rst  |    3 +
 Objects/obmalloc.c                                 | 1070 ++++----------------
 PCbuild/pythoncore.vcxproj                         |    3 +
 PCbuild/pythoncore.vcxproj.filters                 |    9 +
 Python/pylifecycle.c                               |    2 +-
 Python/pystate.c                                   |    7 +-
 Tools/c-analyzer/cpython/globals-to-fix.tsv        |   15 +-
 14 files changed, 1104 insertions(+), 900 deletions(-)
 create mode 100644 Include/internal/pycore_obmalloc.h
 create mode 100644 Include/internal/pycore_obmalloc_init.h
 create mode 100644 Include/internal/pycore_pymem_init.h
 create mode 100644 Misc/NEWS.d/next/Core and Builtins/2022-11-07-14-16-59.gh-issue-81057.3uKlLQ.rst

So the culprit seems to be from this PR

from two years ago, by @ericsnowcurrently

Full git-bisect log:

jdanek@fedora:~/repos/cpython$ git bisect start good v3.11.0 bad main
status: waiting for both good and bad commits
jdanek@fedora:~/repos/cpython$ git bisect good v3.11.0
status: waiting for bad commit, 1 good commit known
jdanek@fedora:~/repos/cpython$ git bisect bad main
Bisecting: a merge base must be tested
[8d32a5c8c4e9c90b0a21e8b2fa0cd567465ced2b] Update the SOURCE_URI in pyspecific.py to point to the new branch
jdanek@fedora:~/repos/cpython$ git bisect good
No testable commit found.
Maybe you started with bad path arguments?
jdanek@fedora:~/repos/cpython$ git bisect start
Previous HEAD position was 8d32a5c8c4 Update the SOURCE_URI in pyspecific.py to point to the new branch
HEAD is now at deaf509e8f Python 3.11.0
status: waiting for both good and bad commits
jdanek@fedora:~/repos/cpython$ git bisect good 8d32a5c8c4e9c90b0a21e8b2fa0cd567465ced2b
status: waiting for bad commit, 1 good commit known
jdanek@fedora:~/repos/cpython$ git bisect bad main 
Bisecting: 3404 revisions left to test after this (roughly 12 steps)
[accb417c338630ac6e836a5c811a89d54a3cd1d3] Replace Netlify with Read the Docs build previews (#103843)
jdanek@fedora:~/repos/cpython$ git bisect skip
Bisecting: 3402 revisions left to test after this (roughly 12 steps)
[f186557dc3e77495ba65b9b7e492ab6ddb7cecc3] gh-103977: compile re expressions in platform.py only if required (#103981)
jdanek@fedora:~/repos/cpython$ git bisect skip
Bisecting: 3402 revisions left to test after this (roughly 12 steps)
[91441bf7cbaefbd328ee7efa59a06e661f61542e] Docs: fix formatting in 2023-06-09-12-59-18 NEWS item (#105607)
jdanek@fedora:~/repos/cpython$ git bisect bad
Bisecting: 2026 revisions left to test after this (roughly 11 steps)
[8dbe08eb7c807f484fe9870f5b7f5ae2881fd966] gh-99240: Fix double-free bug in Argument Clinic str_converter generated code (GH-99241)
jdanek@fedora:~/repos/cpython$ git bisect bad
Bisecting: 1012 revisions left to test after this (roughly 10 steps)
[2fa03b1b0708d5d74630c351ec9abd2aac7550da] gh-95231: Disable md5 & crypt modules if FIPS is enabled (GH-94742)
jdanek@fedora:~/repos/cpython$ git bisect good
Bisecting: 505 revisions left to test after this (roughly 9 steps)
[187e853690908ca2af19a0701ca7529b43d05df9] gh-83940: os docs: Improve wording for getenv/getenvb (#98113)
jdanek@fedora:~/repos/cpython$ git bisect good
Bisecting: 252 revisions left to test after this (roughly 8 steps)
[cff1c20667f668f20be9377ba2bd8d1dc9d56bec] gh-94199: Remove ssl.wrap_socket() documentation (#99023)
jdanek@fedora:~/repos/cpython$ git bisect good
Bisecting: 125 revisions left to test after this (roughly 7 steps)
[504e12272b19395a3c59631a20a94e3323af78e5] GH-81057: Remove static state from arraymodule (#99409)
jdanek@fedora:~/repos/cpython$ git bisect bad
Bisecting: 62 revisions left to test after this (roughly 6 steps)
[c32bc1bffd9d63ede0d0505abab983247a3ad0c6] gh-99124: use concrete exception types in `test_builtin` (#99125)
jdanek@fedora:~/repos/cpython$ git bisect good
Bisecting: 30 revisions left to test after this (roughly 5 steps)
[1e197e63e21f77b102ff2601a549dda4b6439455] GH-96421: Insert shim frame on entry to interpreter (GH-96319)
jdanek@fedora:~/repos/cpython$ git bisect good
Bisecting: 14 revisions left to test after this (roughly 4 steps)
[67b4d2772c5124b908f8ed9b13166a79bbeb88d2] gh-98086: Now ``patch.dict`` can decorate async functions (#98095)
jdanek@fedora:~/repos/cpython$ git bisect good
Bisecting: 6 revisions left to test after this (roughly 3 steps)
[dd36b71fa6164ebba5d94bb4a24eac43b1c54906] gh-81057: Move the Extension Modules Cache to _PyRuntimeState (gh-99355)
jdanek@fedora:~/repos/cpython$ git bisect good
Bisecting: 3 revisions left to test after this (roughly 2 steps)
[67807cfc87135fdce4992d38d2ffe3e44747e73b] gh-81057: Move the Allocators to _PyRuntimeState (gh-99217)
jdanek@fedora:~/repos/cpython$ git bisect bad
Bisecting: 0 revisions left to test after this (roughly 1 step)
[55c96e8053689c29ae28a9d2117ae37934eace68] gh-99305: Speed up secrets.token_hex() ~2x (#99306)
jdanek@fedora:~/repos/cpython$ git bisect good
67807cfc87135fdce4992d38d2ffe3e44747e73b is the first bad commit

@neonene neonene mentioned this issue May 6, 2024
Open
25 tasks
@ericsnowcurrently
Copy link
Member

This should be resolved in main and in the 3.13 branch, since we did the work to isolate _ctypes to each interpreter. Fixing this in 3.12 will be a different matter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.12 bugs and security fixes topic-ctypes topic-subinterpreters type-crash A hard crash of the interpreter, possibly with a core dump
Projects
Subinterpreters
Status: Todo
Milestone
No milestone
Development

No branches or pull requests
5 participants
@jiridanek @ericsnowcurrently @serhiy-storchaka @sobolevn @pablogsal