GC aborts in debug no-gil build

[devdanzin]

Crash report

What happened?

In debug no-gil builds with PYTHON_GIL=0, it's possible to trigger the three assertions here with simple code:

cpython/Python/gc_free_threading.c

Lines 550 to 558 in c84a136

	_PyObject_ASSERT_WITH_MSG(op, !gc_is_unreachable(op),
	"object should not be marked as unreachable yet");
	if (_Py_REF_IS_MERGED(op->ob_ref_shared)) {
	_PyObject_ASSERT_WITH_MSG(op, op->ob_tid == 0,
	"merged objects should have ob_tid == 0");
	}
	else if (!_Py_IsImmortal(op)) {
	_PyObject_ASSERT_WITH_MSG(op, op->ob_tid != 0,

For this code:

import gc
gc.freeze()
gc.is_finalized(lambda: None)
gc.collect()
gc.unfreeze()
gc.collect()

We get:

Python/gc_free_threading.c:550: validate_refcounts: Assertion "!gc_is_unreachable(op)" failed: object should not be marked as unreachable yet
Enable tracemalloc to get the memory block allocation traceback

object address  : 0x200007373f0
object refcount : 1152921504606846977
object type     : 0x55adadc26660
object type name: dict
object repr     : {'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <_frozen_importlib_external.SourceFileLoader object at 0x20000330a20>, '__spec__': None, '__builtins__': <module 'builtins' (built-in)>, '__file__': '/home/fusil/python-61/gc-assertion-abort-2/source.py', '__cached__': None, 'gc': <module 'gc' (built-in)>}

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized

Current thread 0x00007f1b47a64740 (most recent call first):
  Garbage-collecting
  File "/home/fusil/python-61/gc-assertion-abort-2/source.py", line 6 in <module>
Aborted

For this code:

from threading import Thread
import gc

gc.freeze()
alive = [Thread(target=gc.freeze, args=())]
gc.collect()
gc.unfreeze()
gc.collect()

We get:

Python/gc_free_threading.c:554: validate_refcounts: Assertion "op->ob_tid == 0" failed: merged objects should have ob_tid == 0
Enable tracemalloc to get the memory block allocation traceback

object address  : 0x20000278210
object refcount : 1152921504606846994
object type     : 0x5616274befc0
object type name: type
object repr     : <class '_thread.lock'>

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized

Current thread 0x00007f34ed160740 (most recent call first):
  Garbage-collecting
  File "/home/fusil/python-104/gc-abort-assertion/source.py", line 8 in <module>
Aborted

Lastly, for code I haven't minimized yet (but can, if it helps), we get:

Python/gc_free_threading.c:558: validate_refcounts: Assertion "op->ob_tid != 0" failed: unmerged objects should have ob_tid != 0
Enable tracemalloc to get the memory block allocation traceback

object address  : 0x200007980b0
object refcount : 1152921504606846980
object type     : 0x5637720b4f40
object type name: builtin_function_or_method
object repr     : <built-in function collect>

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized

Current thread 0x00007f63aef4c740 (most recent call first):
  Garbage-collecting
  File "/home/fusil/python-106/gc-abort-assertion-sigabrt-37/source.py", line 38 in callMethod
  File "/home/fusil/python-106/gc-abort-assertion-sigabrt-37/source.py", line 42 in callFunc
  File "/home/fusil/python-106/gc-abort-assertion-sigabrt-37/source.py", line 247 in <module>
Aborted

Found using fusil by @vstinner.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.0a1+ experimental free-threading build (heads/main:d467d9246c, Nov 1 2024, 09:05:56) [GCC 11.4.0]

Linked PRs

• gh-126312: Don't traverse frozen objects on the free-threaded build #126338
• [3.13] gh-126312: Don't traverse frozen objects on the free-threaded build (GH-126338) #126866

[ZeroIntensity]

Confirmed on the main branch and 3.13. Interestingly, this doesn't seem to happen on any other objects passed to is_finalized, and not even objects of the same type--only lambdas cause this. I did see that the reference count of a lambda is significantly high, but I'm guessing that's just a result of the DRC bits. I'm investigating to see what's going on.

[colesbury]

The easiest fix would be to disallow gc.freeze() while a GC is in progress.

[ZeroIntensity]

That doesn't seem to fix it on my end. It looks like freezing is entirely broken on the free-threaded builds:

import gc
gc.freeze()
0/0  # Abort

[rruuaanng]

I will test it and try to fix it when I have time. (This does not affect other people to fix it.)

[ZeroIntensity]

Ah, I found the problem. The free-threaded GC only ignores a block if that object itself is frozen, but triggering the tp_traverse of a non-frozen object can traverse over a frozen one, and that breaks its reference count. I've created #126338 as a fix.

[ZeroIntensity]

So, there are actually two bugs at play here. One is the problem that I described above, but then another is that objects that have deferred reference counting enabled seem to have some weird issues when getting frozen. A small reproducer for that:

import gc
import unittest


class Test(unittest.TestCase):
    def test_something(self):
        gc.freeze()
        gc.collect()
        gc.unfreeze()


if __name__ == "__main__":
    unittest.main()

I'd like my PR to land first before fixing that.

Edit: OK, nevermind. As it turns out, the fix was easy. My PR now fixes freezing for DRC as well.