Common issues with extensions

This page lists some of the common issues encountered when building extensions for the nogil fork.

Refcounting tests break

The nogil fork marks some objects as "immortal", including static types, small integers, and interned strings. The reference count of these objects never change. The sys.getrefcount function reports the reference count of these objects as 999. Tests that rely on precise reference counts may not work properly with these objects.

Note: immortal objects are likely to be implemented upstream in CPython independent of the "nogil" fork. See PEP 683.

Symptoms:

• Unexpected refcount of 999 (or nearby number)
• Off by one errors in reference counting tests

Crash due to NULL PyThreadState

CPython requires developers to hold the GIL before calling into the C API. The nogil fork removes the GIL, but still requires developers to call those same functions at the same places. In other words, if you call into Python from a thread created in C code, you still need to call PyGILState_Ensure or similar function. Although these functions no longer prevent concurrency, they are still important for controlling when the garbage collector can run.

Additionally, you should still call the functions/macros that "release the GIL" before calling C or C++ functions that may block the thread, such as I/O or acquiring a C/C++ mutex.

Relevant functions: PyGILState_Ensure, PyGILState_Release PyEval_SaveThread, PyEval_RestoreThread, Py_BEGIN_ALLOW_THREADS, Py_END_ALLOW_THREADS.

Symptoms:

• Segmentation fault in PyErr_Occurred
• _Py_current_tstate is NULL

Python errors due to bytecode format

The nogil fork changes the bytecode format. It introduces new opcodes and removes some opcodes, although many of the same basic opcodes remain.

Python error when using co_names

The nogil fork makes some changes to the code object. The most common issue is due to merging co_names and co_consts. (All constants are held in a single array and co_names is just an alias for co_consts.) In particular, looping over code.co_names may include constants that are not names or even strings.