test__xxsubinterpreters is Occasionally Crashing #104341

ericsnowcurrently · 2023-05-09T22:26:37Z

This appears to have started once gh-104210 was merged.

"Fatal Python error: drop_gil: PyMUTEX_LOCK(gil->switch_mutex) failed"

It may be specific to test.test__xxsubinterpreters.RunStringTests.test_create_thread.

See:

Linked PRs

The text was updated successfully, but these errors were encountered:

ericsnowcurrently · 2023-05-09T22:42:26Z

CC @brandtbucher

ericsnowcurrently · 2023-05-09T22:43:14Z

I've been able to reproduce the crash locally with ./python -m test -j8 test__xxsubinterpreters --forever

brandtbucher · 2023-05-09T23:13:53Z

Core was generated by `./python -m test --forever test__xxsubinterpreters -m test_create_thread'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  raise (sig=sig@entry=11) at ../sysdeps/unix/sysv/linux/raise.c:50
50      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
[Current thread is 1 (Thread 0x7f44daa8a700 (LWP 23889))]
(gdb) bt
#0  raise (sig=sig@entry=11) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x000055d7555a0d94 in faulthandler_fatal_error (signum=11) at ./Modules/faulthandler.c:330
#2  <signal handler called>
#3  drop_gil (ceval=0x7f44dae104d8, tstate=tstate@entry=0x55d756081200) at Python/ceval_gil.c:301
#4  0x000055d75552fad5 in _PyEval_ReleaseLock (tstate=tstate@entry=0x55d756081200) at Python/ceval_gil.c:629
#5  0x000055d75556a9bd in _PyThreadState_DeleteCurrent (tstate=tstate@entry=0x55d756081200) at Python/pystate.c:1568
#6  0x000055d755615cb8 in thread_run (boot_raw=boot_raw@entry=0x7f44dab23790) at ./Modules/_threadmodule.c:1097
#7  0x000055d755587672 in pythread_wrapper (arg=<optimized out>) at Python/thread_pthread.h:233
#8  0x00007f44dc5af609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#9  0x00007f44dc37c293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

brandtbucher · 2023-05-09T23:14:35Z

Also when running test_threads_join and test_threads_join_2 in test_threading with the same flags.

ericsnowcurrently · 2023-05-10T21:43:12Z

I think I've got this figured out. It looks like the interpreter is getting finalized before the thread is completely done in thread_run() (in _threadmodule.c). Apparently the thread isn't completely finalized at the point thread.join() returns. Thus there's a small race there where the runtime may think the thread is done running. It proceeds to clean up the thread state even though the thread is still cleaning itself up.

Based on the above, there are two points at which it should have waited:

thread.join()
wait_for_thread_shutdown() via Py_EndInterpreter() (in pylifecycle.c)

I'm going to figure out why things didn't wait properly. I suspect it looks something like this:

the sub-thread finishes
it "marks" itself as done
it releases the subinterpreter GIL
the subinterpreter's initial thread acquires the GIL
it finishes running
it releases its GIL
the sub-thread acquires the subinterpreter GIL
it finishes cleaning up its thread state
the main interpreter starts running again (with its own GIL)
it calls Py_EndInterpreter()
the thread is already off the "known threads" list so it doesn't wait there
it clears/frees the subinterpreter's remaining thread states and then the subinterpreter itself

The race lies between steps (8) and (12). If (8) doesn't finish before (12) happens then we'd see the crash we're seeing.

What I don't understand is:

what do we need to do differently in step (2)?
when is step (3) happening?

To be honest, I still haven't figured out what we are actually doing in (2) such that it affects thread.join() and wait_for_thread_shutdown(). However that may be irrelevant. I suspect we are missing a step before (9) where we acquire the subinterpreter's GIL while we're cleaning things up in Py_EndInterpreter().

FYI, this doesn't happen when the GIL is shared because (9) is blocked by the GIL being held still in the sub-thread.

ericsnowcurrently · 2023-05-12T13:55:20Z

This may also be related to (or a variant of) gh-96387.

ericsnowcurrently · 2023-05-12T17:31:01Z

Hmm, looks like I need to add _PyInterpreterState_GetFinalizing(), etc. and use it in tstate_must_exit() (in ceval_gil.c).

ericsnowcurrently · 2023-05-12T18:50:02Z

FYI, the two fixes I have up for review don't solve the crashes yet, but I'm getting closer.

ericsnowcurrently · 2023-05-12T23:34:36Z

Currently here's where things are at:

when the sub-thread in test_create_thread() finishes, thread_run() (in _threadmodule.c) starts cleaning up
the last thing it does is call _PyThreadState_DeleteCurrent() (pystate.c), which calls _PyEval_ReleaseLock() (ceval_gil.c)
that turns into drop_gil() (ceval_gil.c), where everything goes smoothly, until...
MUTEX_UNLOCK(gil->mutex); successfully releases the lock, but often gets stuck when there are many processes running the test
it finally unblocks at some point during the call in Py_EndInterpreter() (pylifecycle.c), where the main interpreter is destroying the subinterpreter, likely around the point the subinterpreter's GIL is destoyed
drop_gil() subsequently blows up because the thread state is no longer valid

Here's a detailed breakdown of execution:

(expand)

Fatal Python error: Segmentation fault

Thread 0x00007fec567fc700 (most recent call first):
  File "/home/esnow/projects/work/cpython-perf/cpython/Lib/subprocess.py", line 1971 in _try_wait
  File "/home/esnow/projects/work/cpython-perf/cpython/Lib/subprocess.py", line 2013 in _wait
  File "/home/esnow/projects/work/cpython-perf/cpython/Lib/subprocess.py", line 1262 in wait
  File "/home/esnow/projects/work/cpython-perf/cpython/Lib/test/libregrtest/runtest_mp.py", line 241 in _run_process
  File "/home/esnow/projects/work/cpython-perf/cpython/Lib/test/libregrtest/runtest_mp.py", line 293 in _runtest
  File "/home/esnow/projects/work/cpython-perf/cpython/Lib/test/libregrtest/runtest_mp.py", line 342 in run
  File "/home/esnow/projects/work/cpython-perf/cpython/Lib/threading.py", line 1056 in _bootstrap_inner
  File "/home/esnow/projects/work/cpython-perf/cpython/Lib/threading.py", line 1013 in _bootstrap

0:00:00 load avg: 0.29 [  1/1] test__xxsubinterpreters crashed (Exit code -11)
Thread 0x00007fec56ffd700 (most recent call first):
  File "/home/esnow/projects/work/cpython-perf/cpython/Lib/subprocess.py ", line 1971 in _try_wait
  File "/home/esnow/projects/work/cpython-perf/cpython/Lib/subprocess.py", line 2013 in _wait
  File "/home/esnow/projects/work/cpython-perf/cpython/Lib/subprocess.py", line 1262 in wait

   28237 drop_gil                       ts:0x561957657b40 (cur:0x561957657b40) - is:0x5619575f9660 gil:0x561957657a70 <entered>
   28237 drop_gil                       ts:0x561957657b40 (cur:0x561957657b40) - is:0x5619575f9660 gil:0x561957657a70 starting
   28237 drop_gil                       ts:0x561957657b40 (cur:0x561957657b40) - is:0x5619575f9660 gil:0x561957657a70 done
   28237 drop_gil                       ts:0x561957657b40 (cur:0x561957657b40) - is:0x5619575f9660 gil:0x561957657a70 returning
   28237 take_gil                       ts:0x561957657b40 (cur:0x561957657b40) - is:0x5619575f9660 gil:0x561957657a70 <entered>
   28237 take_gil                       ts:0x561957657b40 (cur:0x561957657b40) - is:0x5619575f9660 gil:0x561957657a70 acquired mutex
   28237 take_gil                       ts:0x561957657b40 (cur:0x561957657b40) - is:0x5619575f9660 gil:0x561957657a70 aquired (locked)
   28237 take_gil                       ts:0x561957657b40 (cur:0x561957657b40) - is:0x5619575f9660 gil:0x561957657a70 released mutex
   28237 drop_gil                       ts:0x561957657b40 (cur:0x561957657b40) - is:0x5619575f9660 gil:0x561957657a70 <entered>
   28237 drop_gil                       ts:0x561957657b40 (cur:0x561957657b40) - is:0x5619575f9660 gil:0x561957657a70 starting
   28237 drop_gil                       ts:0x561957657b40 (cur:0x561957657b40) - is:0x5619575f9660 gil:0x561957657a70 done
   28237 drop_gil                       ts:0x561957657b40 (cur:0x561957657b40) - is:0x5619575f9660 gil:0x561957657a70 returning

############################################

28247 starting test
1
2
28247 running script
28247 starting thread
   28247 thread_run               XXXXX ts:0x55d128cf2f00 (cur:(nil)) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 thread_run               XXXXX ts:0x55d128cf2f00 (cur:(nil)) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquiring GIL
   28247 take_gil                 XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                 XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                 XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                 XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (last holder)
   28247 take_gil                 XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 thread_run               XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired GIL
   28247 thread_run               XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 about to run
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 thread_run               XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 finishing
   28247 thread_run               XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 almost done
   28247 _PyThreadState_DeleteCurrent X ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 _PyThreadState_DeleteCurrent X ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 releasing GIL
   28247 drop_gil                 XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                 XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                 XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 drop_gil                 XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 locked = 1
   28247 drop_gil                 XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 signaled cond
          ================= should unlock mutex here ==================
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released/acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (last holder)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
28247 joining
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
28247 joined
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
28247 script finished

############################################

(decref) interpid 2 (0)
finalizing 2
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 Py_EndInterpreter              ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
28247 shutting down threads
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
28247 stopping main thread
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 starting
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 <entered>
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 acquired mutex
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 aquired (locked)
   28247 take_gil                       ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released mutex
   28247 Py_EndInterpreter              ts:0x7f69e1a6a500 (cur:0x7f69e1a6a500) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 clearing subinterpreter
   28247 drop_gil                 XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 done
   28247 drop_gil                 XXXXX ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 returning
   28247 _PyThreadState_DeleteCurrent X ts:0x55d128cf2f00 (cur:0x55d128cf2f00) - is:0x7f69e1a0c020 gil:0x7f69e1a6a430 released GIL
Fatal Python error: Segmentation fault

ericsnowcurrently · 2023-05-12T23:40:36Z

@pitrou, any ideas on this?

ericsnowcurrently · 2023-05-13T16:17:41Z

Might this be a consequence of thread scheduling? When the sub-thread releases the GIL mutex in drop_gil(), perhaps the thread doesn't get re-scheduled by the kernel until the subintereter's GIL is destroyed? I expect that would be with MUTEX_FINI() during the destroy_gil() call in ceval_gil.c.

That doesn't seem right, but I'm not very experienced with how the pthread APIs and OS thread scheduling works. If it is right, what's the best way to make sure the sub-thread gets scheduled right away?

ericsnowcurrently · 2023-05-13T18:43:58Z

gh-63008 is looking awfully related!

gh-18296 also plays a part.

sunmy2019 · 2023-05-15T13:00:39Z

It has occurred very often recently.

We got the hypothesis test (Ubuntu) failure on the 6/10 most recent commits on main.
https://github.com/python/cpython/actions/runs/4978751491/jobs/8909463684

sunmy2019 · 2023-05-15T13:52:40Z

I cannot reproduce them on my machine (WSL Ubuntu 22.04, AMD 7950X, gcc-11 and gcc-9)

I also rent a AMD VPS (Ubuntu 22.04, gcc-11), still cannot reproduce with either

./python -m test -j8 test__xxsubinterpreters --forever

or

./python -m test --forever test__xxsubinterpreters -m test_create_thread

…ion (gh-104437) With the move to a per-interpreter GIL, this check slipped through the cracks.

* main: pythonGH-104510: Fix refleaks in `_io` base types (python#104516) pythongh-104539: Fix indentation error in logging.config.rst (python#104545) pythongh-104050: Don't star-import 'types' in Argument Clinic (python#104543) pythongh-104050: Add basic typing to CConverter in clinic.py (python#104538) pythongh-64595: Fix write file logic in Argument Clinic (python#104507) pythongh-104523: Inline minimal PGO rules (python#104524) pythongh-103861: Fix Zip64 extensions not being properly applied in some cases (python#103863) pythongh-69152: add method get_proxy_response_headers to HTTPConnection class (python#104248) pythongh-103763: Implement PEP 695 (python#103764) pythongh-104461: Run tkinter test_configure_screen on X11 only (pythonGH-104462) pythongh-104469: Convert _testcapi/watchers.c to use Argument Clinic (python#104503) pythongh-104482: Fix error handling bugs in ast.c (python#104483) pythongh-104341: Adjust tstate_must_exit() to Respect Interpreter Finalization (pythongh-104437) pythonGH-102613: Fix recursion error from `pathlib.Path.glob()` (pythonGH-104373)

ericsnowcurrently · 2023-05-16T19:42:59Z

My fix in gh-104560 should take care of this.

ericsnowcurrently · 2023-05-19T20:59:52Z

The fix in gh-104672 should take care of the buildbot failures. I'll probably need to update the docs for PyThreadState_Delete(), which references "on_delete", but I can circle back to that after the fix is in.

Having a separate lock means Thread.join() doesn't need to wait for the thread to be cleaned up first. It can wait for the thread's Python target to finish running. This gives us some flexibility in how we clean up threads. (This is a minor cleanup as part of a fix for gh-104341.)

ericsnowcurrently · 2023-05-23T22:02:24Z

Also relevant: gh-84191, gh-84165, and gh-77789.

…ck for Each Thread (pythongh-104754)" This reverts commit 097b783.

…Thread (gh-104754) (#104838) gh-104837: Revert "gh-104341: Add a Separate "Running" Lock for Each Thread (gh-104754)" This reverts commit 097b783.

ericsnowcurrently · 2023-05-30T22:06:40Z

After a conversation with @markshannon, I had an epiphany about how to solve this minimally: gh-105109.

pitrou · 2023-05-31T08:41:43Z

It seems each interpreter could maintain its current number of threads? It would be decremented at the very end of thread_run after everything else ran, giving Py_EndInterpreter a reliable way of knowing that there's nothing going on anymore.

ericsnowcurrently · 2023-05-31T21:20:26Z

Yeah, that makes sense.

ericsnowcurrently · 2023-05-31T23:28:09Z

Hmm, daemon threads would be a problem still, no?

gpshead · 2023-06-01T00:15:17Z

We should just disallow daemon threads from Subinterpreters. (Silently or loudly, no opinion)

ericsnowcurrently · 2023-06-01T00:26:44Z

We already do by default for interpreters created with Py_NewInterpreterFromConfig(), but now with the old Py_NewInterpreter().

@markshannon

…urrent Thread (gh-105109) This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads. (The idea for this approach came out of discussions with @markshannon.)

@markshannon

… the Current Thread (pythongh-105109) This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads. (The idea for this approach came out of discussions with @markshannon.) (cherry picked from commit 3698fda) Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>

@markshannon

…g the Current Thread (gh-105109) (gh-105209) This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads. (The idea for this approach came out of discussions with @markshannon.) (cherry picked from commit 3698fda) Co-authored-by: Eric Snow ericsnowcurrently@gmail.com

ericsnowcurrently · 2023-06-01T23:35:41Z

I've merged my fix to main and 3.12. We should be good to go now. FYI, I do plan on circling back to the underlying problem in the 3.13 timeframe.

ericsnowcurrently · 2023-06-02T16:37:40Z

FYI, there are other crashes I know about but they are much less common and have a different root cause.

Regarding the additional work I mentioned, I'll open a new issue once I get around to it.

ericsnowcurrently added interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump topic-subinterpreters 3.12 bugs and security fixes labels May 9, 2023

ericsnowcurrently mentioned this issue May 9, 2023

GH-104308: socket.getnameinfo should release the GIL #104307

Merged

ericsnowcurrently mentioned this issue May 12, 2023

gh-104341: Ensure tstate Holding GIL is Always Current #104436

Draft

bedevere-bot mentioned this issue May 12, 2023

gh-104341: Adjust tstate_must_exit() to Respect Interpreter Finalization #104437

Merged

AlexWaygood mentioned this issue May 13, 2023

GH-86275: Implementation of hypothesis stubs for property-based tests, with zoneinfo tests #22863

Merged

This was referenced May 13, 2023

gh-104454: Fix refleak in AttributeError_reduce #104455

Merged

gh-87646: Added .path to tempfile #104466

Open

ericsnowcurrently added a commit that referenced this issue May 15, 2023

gh-104341: Adjust tstate_must_exit() to Respect Interpreter Finalizat…

26baa74

…ion (gh-104437) With the move to a per-interpreter GIL, this check slipped through the cracks.

ericsnowcurrently mentioned this issue May 16, 2023

gh-104341: Clean Up threading Module #104552

Closed

bedevere-bot mentioned this issue May 16, 2023

gh-104341: Fix threading Module Shutdown #104560

Draft

melwitt mentioned this issue May 16, 2023

gh-85934: Use getattr_static when adding mock spec #22209

Merged

This was referenced May 22, 2023

gh-104341: Add a Separate "Running" Lock for Each Thread #104754

Merged

gh-104812: Run Pending Calls in any Thread #104813

Merged

miss-islington mentioned this issue May 23, 2023

[3.12] gh-104341: Add a Separate "Running" Lock for Each Thread (gh-104754) #104817

Closed

gpshead added a commit to gpshead/cpython that referenced this issue May 24, 2023

pythongh-104837: Revert "pythongh-104341: Add a Separate "Running" Lo…

5ceae3c

…ck for Each Thread (pythongh-104754)" This reverts commit 097b783.

bedevere-bot mentioned this issue May 30, 2023

gh-104341: Call _PyEval_ReleaseLock() with NULL When Finalizing the Current Thread #105109

Merged

ericsnowcurrently added the 3.13 bugs and security fixes label May 30, 2023

bedevere-bot mentioned this issue Jun 1, 2023

[3.12] gh-104341: Call _PyEval_ReleaseLock() with NULL When Finalizing the Current Thread (gh-105109) #105209

Merged

ericsnowcurrently closed this as completed Jun 2, 2023

This was referenced Jun 5, 2023

Threading running lock #105334

Closed

Segmentation fault during the test__xxsubinterpreters test #100981

Open

Race condition in subinterpreters during subinterpreter creation on Windows debug build #100711

Open

erlend-aasland mentioned this issue Jun 6, 2023

3.12 backport gh 105236 #105358

Closed

ericsnowcurrently mentioned this issue Oct 25, 2023

threading Thread.join should call the OS join API #110829

Closed

neonene mentioned this issue Aug 13, 2024

asserting that the tstate pointer is non-NULL using assert #122928

Closed

gpshead mentioned this issue Oct 2, 2024

race condition in threading when interpreter finalized while daemon thread runs (thread sanitizer identified) #124878

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test__xxsubinterpreters is Occasionally Crashing #104341

test__xxsubinterpreters is Occasionally Crashing #104341

ericsnowcurrently commented May 9, 2023 •

edited by bedevere-bot

Loading

ericsnowcurrently commented May 9, 2023

ericsnowcurrently commented May 9, 2023

brandtbucher commented May 9, 2023

brandtbucher commented May 9, 2023

ericsnowcurrently commented May 10, 2023

ericsnowcurrently commented May 12, 2023

ericsnowcurrently commented May 12, 2023

ericsnowcurrently commented May 12, 2023

ericsnowcurrently commented May 12, 2023

ericsnowcurrently commented May 12, 2023

ericsnowcurrently commented May 13, 2023

ericsnowcurrently commented May 13, 2023 •

edited

Loading

sunmy2019 commented May 15, 2023

sunmy2019 commented May 15, 2023

ericsnowcurrently commented May 16, 2023

ericsnowcurrently commented May 19, 2023

ericsnowcurrently commented May 23, 2023

ericsnowcurrently commented May 30, 2023

pitrou commented May 31, 2023

ericsnowcurrently commented May 31, 2023

ericsnowcurrently commented May 31, 2023

gpshead commented Jun 1, 2023

ericsnowcurrently commented Jun 1, 2023

ericsnowcurrently commented Jun 1, 2023

ericsnowcurrently commented Jun 2, 2023

test__xxsubinterpreters is Occasionally Crashing #104341

test__xxsubinterpreters is Occasionally Crashing #104341

Comments

ericsnowcurrently commented May 9, 2023 • edited by bedevere-bot Loading

Linked PRs

ericsnowcurrently commented May 9, 2023

ericsnowcurrently commented May 9, 2023

brandtbucher commented May 9, 2023

brandtbucher commented May 9, 2023

ericsnowcurrently commented May 10, 2023

ericsnowcurrently commented May 12, 2023

ericsnowcurrently commented May 12, 2023

ericsnowcurrently commented May 12, 2023

ericsnowcurrently commented May 12, 2023

ericsnowcurrently commented May 12, 2023

ericsnowcurrently commented May 13, 2023

ericsnowcurrently commented May 13, 2023 • edited Loading

sunmy2019 commented May 15, 2023

sunmy2019 commented May 15, 2023

ericsnowcurrently commented May 16, 2023

ericsnowcurrently commented May 19, 2023

ericsnowcurrently commented May 23, 2023

ericsnowcurrently commented May 30, 2023

pitrou commented May 31, 2023

ericsnowcurrently commented May 31, 2023

ericsnowcurrently commented May 31, 2023

gpshead commented Jun 1, 2023

ericsnowcurrently commented Jun 1, 2023

ericsnowcurrently commented Jun 1, 2023

ericsnowcurrently commented Jun 2, 2023

ericsnowcurrently commented May 9, 2023 •

edited by bedevere-bot

Loading

ericsnowcurrently commented May 13, 2023 •

edited

Loading