Segfault in invalid concurrent.futures.interpreter.WorkerContext

[devdanzin]

Crash report

What happened?

It's possible to segfault the interpreter by calling initialize() on a concurrent.futures.interpreter.WorkerContext instance that was created with the shared argument being a dict containing the null byte as a key:

python -c "import concurrent.futures.interpreter; w = concurrent.futures.interpreter.WorkerContext(0, {'\x00': ''}).initialize()"

This doesn't require threads or free-threading. It can be traced to the _interpreters module:

import _interpreters
_interpreters.create()
_interpreters.set___main___attrs(1, {"\x00": 1}, restrict=True)
Segmentation fault (core dumped)

The backtrace is:

#0  Py_INCREF (op=op@entry=0x0) at ./Include/refcount.h:241
#1  0x00005555557e6aa7 in _Py_NewRef (obj=0x0) at ./Include/refcount.h:492
#2  _sharednsitem_apply (item=0x555555cd55e0, ns=ns@entry=0x200028329b0, dflt=dflt@entry=0x0) at Python/crossinterp.c:1224
#3  0x00005555557e7d14 in _PyXI_ApplyNamespace (ns=ns@entry=0x555555cd55b0, nsobj=nsobj@entry=0x200028329b0, dflt=dflt@entry=0x0) at Python/crossinterp.c:1523
#4  0x00005555557e7ec8 in _PyXI_Enter (session=session@entry=0x7fffffffe040, interp=interp@entry=0x7ffff7bb8020, nsupdates=<optimized out>)
    at Python/crossinterp.c:1754
#5  0x00007ffff7e1fafd in interp_set___main___attrs (self=self@entry=0x20000966980, args=args@entry=0x20000943850, kwargs=kwargs@entry=0x20000736eb0)
    at ./Modules/_interpretersmodule.c:836
#6  0x00005555556c1565 in cfunction_call (func=func@entry=0x20000966830, args=args@entry=0x20000943850, kwargs=kwargs@entry=0x20000736eb0)
    at Objects/methodobject.c:551
#7  0x000055555566987f in _PyObject_MakeTpCall (tstate=tstate@entry=0x555555c39510 <_PyRuntime+360208>, callable=callable@entry=0x20000966830, 
    args=args@entry=0x7fffffffe3c8, nargs=<optimized out>, keywords=keywords@entry=0x2000057e700) at Objects/call.c:242
#8  0x0000555555669ada in _PyObject_VectorcallTstate (tstate=0x555555c39510 <_PyRuntime+360208>, callable=callable@entry=0x20000966830, 
    args=args@entry=0x7fffffffe3c8, nargsf=<optimized out>, kwnames=kwnames@entry=0x2000057e700) at ./Include/internal/pycore_call.h:165
#9  0x0000555555669b30 in PyObject_Vectorcall (callable=callable@entry=0x20000966830, args=args@entry=0x7fffffffe3c8, nargsf=<optimized out>, 
    kwnames=kwnames@entry=0x2000057e700) at Objects/call.c:327
#10 0x00005555557adebb in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=0x7ffff7e6a0a0, throwflag=<optimized out>) at Python/generated_cases.c.h:1982
#11 0x00005555557c755c in _PyEval_EvalFrame (tstate=tstate@entry=0x555555c39510 <_PyRuntime+360208>, frame=<optimized out>, throwflag=throwflag@entry=0)
    at ./Include/internal/pycore_ceval.h:116
#12 0x00005555557c766a in _PyEval_Vector (tstate=tstate@entry=0x555555c39510 <_PyRuntime+360208>, func=func@entry=0x200007b3490, locals=locals@entry=0x20000737870, 
    args=args@entry=0x0, argcount=argcount@entry=0, kwnames=kwnames@entry=0x0) at Python/ceval.c:1898
#13 0x00005555557c7739 in PyEval_EvalCode (co=co@entry=0x200003a3f10, globals=globals@entry=0x20000737870, locals=locals@entry=0x20000737870) at Python/ceval.c:659
#14 0x000055555588bac3 in run_eval_code_obj (tstate=tstate@entry=0x555555c39510 <_PyRuntime+360208>, co=co@entry=0x200003a3f10, globals=globals@entry=0x20000737870, 
    locals=locals@entry=0x20000737870) at Python/pythonrun.c:1338
#15 0x000055555588bca3 in run_mod (mod=mod@entry=0x200007e42b0, filename=filename@entry=0x20000a462f0, globals=globals@entry=0x20000737870, 
    locals=locals@entry=0x20000737870, flags=flags@entry=0x7fffffffe720, arena=arena@entry=0x200000508b0, interactive_src=0x200002338f0, generate_new_source=0)
    at Python/pythonrun.c:1423
#16 0x000055555588c6ad in _PyRun_StringFlagsWithName (
    str=str@entry=0x200002341e0 "import concurrent.futures.interpreter; w = concurrent.futures.interpreter.WorkerContext(0, {'\\x00': ''}).initialize()\n", 
    name=name@entry=0x20000a462f0, start=start@entry=257, globals=globals@entry=0x20000737870, locals=locals@entry=0x20000737870, flags=flags@entry=0x7fffffffe720, 
    generate_new_source=0) at Python/pythonrun.c:1222

Found using fusil by @vstinner.

CPython versions tested on:

3.14, CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.0a2+ (heads/main:0af4ec3, Nov 20 2024, 21:45:19) [GCC 13.2.0]

Linked PRs

• gh-127165: Disallow embedded NULL characters in _interpreters #127199
• [3.13] gh-127165: Disallow embedded NULL characters in _interpreters (GH-127199) #127463

[ZeroIntensity]

I'm assuming this is because of a missing NULL check on PyUnicode_AsUTF8 again. Should be a pretty easy fix, I'll deal with it if nobody else submits a PR.

[rruuaanng]

I'm assuming this is because of a missing NULL check on PyUnicode_AsUTF8 again. Should be a pretty easy fix, I'll deal with it if nobody else submits a PR.

If you can, give it to me :)

[ZeroIntensity]

Apparently, this is because PyUnicode_AsUTF8 doesn't do anything about embedded null characters, so some things in the dictionary lookups break down when converting from C to Python strings.

[vstinner]

Apparently, this is because PyUnicode_AsUTF8 doesn't do anything about embedded null characters

There is a private _PyUnicode_AsUTF8NoNUL() function which rejects null characters.