iter __reduce__ can segfault if accessing __builtins__.__dict__['iter'] mutates the iter object

[ionite34]

Crash report

Example from @chilaxan

corrupt = iter(lambda:0, 0)

class Cstr:
    def __hash__(self):
        return hash('iter')
    def __eq__(self, other):
        [*corrupt]
        return other == 'iter'

builtins = __builtins__.__dict__ if hasattr(__builtins__, '__dict__') else __builtins__
oiter = builtins['iter']
del builtins['iter']
builtins[Cstr()] = oiter

print(corrupt.__reduce__())

Expected result

This should return a valid __reduce__ tuple of the exhausted iterator. Instead behavior is inconsistent between segmentation faults, SystemErrors, and sometimes returning the iterator without being exhausted.

Error messages

• 3.11, windows, PYTHONMALLOC=debug
• 3.12.0a4, windows, PYTHONMALLOC=debug

Windows fatal exception: access violation
> exit code -1073741819 (0xC0000005)

• 3.12.04a4, windows, compiled with debug mode

    print(corrupt.__reduce__())
          ^^^^^^^^^^^^^^^^^^^^
SystemError: NULL object passed to Py_BuildValue

• 3.11, ubuntu

(<built-in function iter>, (<function  at 0x7fb772c3c4a0>, 0))
> terminated by signal SIGSEGV (Address boundary error)

• 3.12.0a4, ubuntu

(<built-in function iter>, (<function  at 0x7f3480d71f80>, 0))

• 3.12.0a4, ubuntu, PYTHONMALLOC=debug

Fatal Python error: Segmentation fault

Linked PRs

• gh-101765: Fix SystemError / segmentation fault in iter __reduce__ when internal access of builtins.__dict__ exhausts the iterator #101769
• [3.11] gh-101765: Fix SystemError / segmentation fault in iter __reduce__ when internal access of builtins.__dict__ exhausts the iterator (GH-101769) #102228
• [3.10] gh-101765: Fix SystemError / segmentation fault in iter __reduce__ when internal access of builtins.__dict__ exhausts the iterator (GH-101769) #102229
• gh-101765: Fix refcount issues in list and unicode pickling #102265
• [3.11] gh-101765: Fix refcount issues in list and unicode pickling (G… #102268
• [3.10] gh-101765: Fix refcount issues in list and unicode pickling (G… #102269
• gh-101765: unicodeobject: use Py_XDECREF correctly #102283
• [3.11] gh-101765: unicodeobject: use Py_XDECREF correctly (GH-102283) #102285
• [3.10] gh-101765: unicodeobject: use Py_XDECREF correctly (GH-102283) #102286

[godlygeek]

This (horrific) reproducer provokes undefined defined behavior from this statement:

cpython/Objects/iterobject.c

Lines 243 to 244 in f1f3af7

	return Py_BuildValue("N(OO)", _PyEval_GetBuiltin(&_Py_ID(iter)),
	it->it_callable, it->it_sentinel);

The call to _PyEval_GetBuiltin to find the iter builtin is calling Cstr.__eq__, which exhausts the iterator, causing the Py_CLEAR in calliter_iternext to be executed, setting it->it_callable and it->it_sentinel to NULL. But the order of evaluation of arguments in a function call isn't specified, and modifying an argument by evaluating another argument is a bug.

On some platforms, it->it_callable and it->it_sentinel are being evaluated before _PyEval_GetBuiltin, and so Py_BuildValue is being passed pointers to objects that got freed inside the _PyEval_GetBuiltin call.