Running `tracemalloc` in threads segfaults on no-gil

[devdanzin]

Crash report

What happened?

This crash only happens on a no-gil build (only debug tested) with PYTHON_GIL=0.

# This segfaults with PYTHON_GIL=0, works with PYTHON_GIL=1
from threading import Thread
import tracemalloc

alive = [
    Thread(target=tracemalloc.take_snapshot, args=()),
    Thread(target=tracemalloc.clear_traces, args=()),
]

tracemalloc.start()
import inspect # This seems to be important, has to happen after .start()

for obj in alive:
    print('START', obj)
    obj.start()

Backtrace looks like:

frame_to_pyobject (frame=frame@entry=0x555555e78eac) at Python/tracemalloc.c:994
994         PyTuple_SET_ITEM(frame_obj, 0, Py_NewRef(frame->filename));
(gdb) bt
#0  frame_to_pyobject (
    frame=frame@entry=0x555555e78eac)
    at Python/tracemalloc.c:994
#1  0x000055555591edbd in traceback_to_pyobject (
    traceback=0x555555e78ea0,
    intern_table=intern_table@entry=0x7ffff00019c0)
    at Python/tracemalloc.c:1024
#2  0x000055555591efee in trace_to_pyobject (
    domain=<optimized out>, trace=0x7ffff000ee70,
    intern_tracebacks=0x7ffff00019c0)
    at Python/tracemalloc.c:1070
#3  0x000055555591f46d in tracemalloc_get_traces_fill (traces=<optimized out>, key=<optimized out>,
    value=<optimized out>,
    user_data=0x7ffff7ba7880)
    at Python/tracemalloc.c:1188
#4  0x00005555558b7f49 in _Py_hashtable_foreach (
    ht=0x7ffff0001ab0,
    func=func@entry=0x55555591f44a <tracemalloc_get_traces_fill>,
    user_data=user_data@entry=0x7ffff7ba7880)
    at Python/hashtable.c:275
#5  0x0000555555920889 in _PyTraceMalloc_GetTraces
    () at Python/tracemalloc.c:1470
#6  0x0000555555957b44 in _tracemalloc__get_traces_impl (module=<optimized out>)
    at ./Modules/_tracemalloc.c:57

Found using fusil by @vstinner.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.0a1+ experimental free-threading build (heads/main:d467d9246c, Nov 1 2024, 09:05:56) [GCC 11.4.0]

[kumaraditya303]

This doesn't crashes on current main branch

[vstinner]

I confirm that I can no longer reproduce the crash in the main branch on Python built with --disable-gil. I close the issue.

I redesigned tracemalloc locking in issue gh-128679 (ex: commit 36c5e3b).