TextIOWrapper.reconfigure() crashes if encoding is not string or None

[serhiy-storchaka]

Crash report

What happened?

Unlike to TextIOWrapper constructor, TextIOWrapper.reconfigure() does not check that the encoding argument is actually None or string. If it is not None, it calls _PyUnicode_EqualToASCIIString that only works with strings and crashes in debug build. There may be other similar errors in other arguments.

>>> import sys
>>> sys.stdout.reconfigure(encoding=42)
Objects/unicodeobject.c:552: _PyUnicode_CheckConsistency: Assertion failed: PyType_HasFeature((Py_TYPE(((PyObject*)((op))))), ((1UL << 28)))
Enable tracemalloc to get the memory block allocation traceback

object address  : 0x55a2e3147ca8
object refcount : 4294967295
object type     : 0x55a2e3119f00
object type name: int
object repr     : 42

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized

Current thread 0x00007fc99ee43740 (most recent call first):
  File "<stdin>-1", line 1 in <module>
Aborted (core dumped)

CPython versions tested on:

3.10, 3.11, 3.12

Operating systems tested on:

No response

Output from running 'python -VV' on the command line:

No response

Linked PRs

• gh-111942: Fix crashes in TextIOWrapper.reconfigure() #111976
• [3.12] gh-111942: Fix crashes in TextIOWrapper.reconfigure() (GH-111976) #112058
• [3.11] gh-111942: Fix crashes in TextIOWrapper.reconfigure() (GH-111976) #112059
• gh-111942: Fix SystemError in the TextIOWrapper constructor #112061
• gh-111942: Fix test_io test_constructor() #112067
• [3.12] gh-111942: Fix SystemError in the TextIOWrapper constructor (GH-112061) #112085
• [3.12] gh-111942: Fix SystemError in the TextIOWrapper constructor (#112061) #112089
• [3.11] gh-111942: Remove an extra incref in textiowrapper_change_encoding #126542

[serhiy-storchaka]

Weird. The address sanitizer tests were passed in main and 3.12, but failed in 3.11. And I now can reproduce the failure locally in main.

The newly added tests exposed a bug in the TextIOWrapper constructor when pass a non-encodable "errors" argument.

[vstinner]

Nice thanks, thanks.

[encukou]

There's an issue with the 3.11 backport, in textiowrapper_change_encoding. Main had:

+    Py_INCREF(errors);
 ...
     Py_SETREF(self->encoding, encoding);
-    Py_SETREF(self->errors, Py_NewRef(errors));
+    Py_SETREF(self->errors, errors);

but in 3.11 this is:

+    Py_INCREF(errors);
...
     Py_INCREF(errors);
     Py_SETREF(self->encoding, encoding);
     Py_SETREF(self->errors, errors);

i.e. there's an extra incref, but it looks -- at least to Git -- like the change that removes Py_NewRef was already applied.

This was not caught because test_io refleaks tests were ineffective on 3.11.

I'm preparing a fix.