Crash for 3.13rc1 on Windows with free threading during initialize

[tonybaloney]

Crash report

What happened?

I'm trying to use Python 3.13rc1 with free-threading and debugging enabled on Windows 11. On startup, Python crashes. I've got a stack trace of the issue, which is pointing to the ob_type being invalid for an object during the startup sequence.

When I run Python at the command line (for anything other than show version) it crashes immediately.

Exception thrown: read access violation.
v->**ob_type** was 0xFFFFFFFFFFFFFF8F.

Full stack trace

python313t_d.dll!binary_op1(_object * v, _object * w, const int op_slot, const char * op_name) Line 934
	at C:\Users\anthonyshaw\source\repos\cpython\Objects\abstract.c(934)
python313t_d.dll!PyNumber_Add(_object * v, _object * w) Line 1141
	at C:\Users\anthonyshaw\source\repos\cpython\Objects\abstract.c(1141)
python313t_d.dll!_PyEval_EvalFrameDefault(_ts * tstate, _PyInterpreterFrame * frame, int throwflag) Line 132
	at C:\Users\anthonyshaw\source\repos\cpython\Python\generated_cases.c.h(132)
python313t_d.dll!_PyEval_Vector(_ts * tstate, PyFunctionObject * func, _object * locals, _object * const * args, unsigned __int64 argcount, _object * kwnames) Line 1806
	at C:\Users\anthonyshaw\source\repos\cpython\Python\ceval.c(1806)
python313t_d.dll!PyEval_EvalCode(_object * co, _object * globals, _object * locals) Line 596
	at C:\Users\anthonyshaw\source\repos\cpython\Python\ceval.c(596)
python313t_d.dll!exec_code_in_module(_ts * tstate, _object * name, _object * module_dict, _object * code_object) Line 2617
	at C:\Users\anthonyshaw\source\repos\cpython\Python\import.c(2617)
python313t_d.dll!PyImport_ImportFrozenModuleObject(_object * name) Line 3048
	at C:\Users\anthonyshaw\source\repos\cpython\Python\import.c(3048)
python313t_d.dll!PyImport_ImportFrozenModule(const char * name) Line 3087
	at C:\Users\anthonyshaw\source\repos\cpython\Python\import.c(3087)
python313t_d.dll!init_importlib(_ts * tstate, _object * sysmod) Line 3167
	at C:\Users\anthonyshaw\source\repos\cpython\Python\import.c(3167)
python313t_d.dll!_PyImport_InitCore(_ts * tstate, _object * sysmod, int importlib) Line 4039
	at C:\Users\anthonyshaw\source\repos\cpython\Python\import.c(4039)
python313t_d.dll!pycore_interp_init(_ts * tstate) Line 904
	at C:\Users\anthonyshaw\source\repos\cpython\Python\pylifecycle.c(904)
python313t_d.dll!pyinit_config(pyruntimestate * runtime, _ts * * tstate_p, const PyConfig * config) Line 933
	at C:\Users\anthonyshaw\source\repos\cpython\Python\pylifecycle.c(933)
python313t_d.dll!pyinit_core(pyruntimestate * runtime, const PyConfig * src_config, _ts * * tstate_p) Line 1096
	at C:\Users\anthonyshaw\source\repos\cpython\Python\pylifecycle.c(1096)
python313t_d.dll!Py_InitializeFromConfig(const PyConfig * config) Line 1398
	at C:\Users\anthonyshaw\source\repos\cpython\Python\pylifecycle.c(1398)
python313t_d.dll!Py_InitializeEx(int install_sigs) Line 1436
	at C:\Users\anthonyshaw\source\repos\cpython\Python\pylifecycle.c(1436)
python313t_d.dll!Py_Initialize() Line 1446
	at C:\Users\anthonyshaw\source\repos\cpython\Python\pylifecycle.c(1446)
[Managed to Native Transition]

CPython versions tested on:

CPython main branch

Operating systems tested on:

Windows

Output from running 'python -VV' on the command line:

Python 3.13.0rc1 experimental free-threading build (tags/v3.13.0rc1:e4a3e786a5e, Aug 12 2024, 08:07:19) [MSC v.1940 64 bit (AMD64)]

[tonybaloney]

I looked at the build to see how it was reproducing workable binaries, but the build for Windows free-threading looks broken?
@DinoV @zooba

[tonybaloney]

This is compiled from source against the 3.13rc1 tag

[colesbury]

We saw these sorts of crashes before when the free-threaded interpreter was loading shared libraries compiled for the default build.

Does the crash go away if you do a "clean" build?

[tonybaloney]

I tried PCBuild\clean.bat first, but that is nowhere near as comprehensive as something like make distclean or make clobber (which doesn't work on Windows).

A fresh clone worked against main and the v3.13.0rc1 tags.

We can either close this or improve clean.bat to remove the libraries that caused this

[colesbury]

Improving the clean.bat script seems like a good idea

[zooba]

clean.bat is going to clean the build configuration that you pass it (same syntax as build.bat), by design.

Adding a distclean.bat that does git clean -xdf might be the best option here.

[ngoldbaum]

I'm seeing similar crashes in my PyO3 dev environment on MacOS. Interestingly, it doesn't happen if I use a debug python 3.13 free-threaded build.

May be completely unrelated to the Windows-specific issues described above.

Here's the traceback from the crash:

running 681 tests
test buffer::tests::test_compatible_size ... ok
test buffer::tests::test_element_type_from_format ... ok
test conversions::std::array::tests::array_try_from_fn ... ok
Process 18512 stopped
* thread #3, name = 'buffer::tests::test_bytes_buffer', stop reason = EXC_BAD_ACCESS (code=2, address=0x1015edb08)
    frame #0: 0x000000010132d048 libpython3.13t.dylib`chacha_block(ctx=0x00000001015edac8) at random.c:67:20 [opt]
   64
   65  	  // add scrambled data to the initial state
   66  	  for (size_t i = 0; i < 16; i++) {
-> 67  	    ctx->output[i] = x[i] + ctx->input[i];
   68  	  }
   69  	  ctx->output_available = 16;
   70
Target 0: (pyo3-71e3a8c0dab6095a) stopped.
warning: libpython3.13t.dylib was compiled with optimization - stepping may behave oddly; variables may not be available.
(lldb) bt
* thread #3, name = 'buffer::tests::test_bytes_buffer', stop reason = EXC_BAD_ACCESS (code=2, address=0x1015edb08)
  * frame #0: 0x000000010132d048 libpython3.13t.dylib`chacha_block(ctx=0x00000001015edac8) at random.c:67:20 [opt]
    frame #1: 0x00000001013218d0 libpython3.13t.dylib`_mi_os_get_aligned_hint [inlined] chacha_next32(ctx=0x00000001015edac8) at random.c:83:5 [opt]
    frame #2: 0x00000001013218bc libpython3.13t.dylib`_mi_os_get_aligned_hint [inlined] _mi_random_next(ctx=0x00000001015edac8) at random.c:149:25 [opt]
    frame #3: 0x00000001013218bc libpython3.13t.dylib`_mi_os_get_aligned_hint [inlined] _mi_heap_random_next(heap=0x00000001015ecf80) at heap.c:258:10 [opt]
    frame #4: 0x00000001013218b8 libpython3.13t.dylib`_mi_os_get_aligned_hint(try_alignment=33554432, size=1073741824) at os.c:118:19 [opt]
    frame #5: 0x000000010132ee50 libpython3.13t.dylib`unix_mmap_prim(addr=0x0000000000000000, size=1073741824, try_alignment=33554432, protect_flags=3, flags=4162, fd=1677721600) at prim.c:190:18 [opt]
    frame #6: 0x0000000101327f68 libpython3.13t.dylib`_mi_prim_alloc [inlined] unix_mmap(addr=0x0000000000000000, size=<unavailable>, try_alignment=<unavailable>, protect_flags=3, large_only=false, allow_large=<unavailable>, is_large=<unavailable>) at prim.c:297:9 [opt]
    frame #7: 0x0000000101327ed4 libpython3.13t.dylib`_mi_prim_alloc(size=1073741824, try_alignment=33554432, commit=<unavailable>, allow_large=<unavailable>, is_large=0x0000000170211bdf, is_zero=<unavailable>, addr=0x0000000170211b78) at prim.c:334:11 [opt]
    frame #8: 0x0000000101321c18 libpython3.13t.dylib`mi_os_prim_alloc(size=1073741824, try_alignment=33554432, commit=true, allow_large=<unavailable>, is_large=<unavailable>, is_zero=<unavailable>, stats=<unavailable>) at os.c:201:13 [opt]
    frame #9: 0x000000010131a648 libpython3.13t.dylib`_mi_os_alloc_aligned [inlined] mi_os_prim_alloc_aligned(size=1073741824, alignment=33554432, commit=true, allow_large=<unavailable>, is_large=0x0000000170211bdf, is_zero=0x0000000170211bde, base=<unavailable>, stats=<unavailable>) at os.c:234:13 [opt]
    frame #10: 0x000000010131a60c libpython3.13t.dylib`_mi_os_alloc_aligned(size=<unavailable>, alignment=33554432, commit=true, allow_large=<unavailable>, memid=0x0000000170211c68, tld_stats=<unavailable>) at os.c:320:13 [opt]
    frame #11: 0x000000010131bb84 libpython3.13t.dylib`mi_reserve_os_memory_ex(size=1073741824, commit=true, allow_large=<unavailable>, exclusive=false, arena_id=0x0000000170211ccc) at arena.c:813:17 [opt]
    frame #12: 0x0000000101319fd0 libpython3.13t.dylib`_mi_arena_alloc_aligned [inlined] mi_arena_reserve(req_size=33554432, allow_large=true, req_arena_id=0, arena_id=0x0000000170211ccc) at arena.c:362:11 [opt]
    frame #13: 0x0000000101319f6c libpython3.13t.dylib`_mi_arena_alloc_aligned(size=33554432, alignment=33554432, align_offset=0, commit=true, allow_large=true, req_arena_id=0, memid=0x0000000170211da8, tld=0x00000001016cc828) at arena.c:383:11 [opt]
    frame #14: 0x000000010132dde8 libpython3.13t.dylib`mi_segment_alloc [inlined] mi_segment_os_alloc(required=0, page_alignment=<unavailable>, eager_delayed=<unavailable>, req_arena_id=0, psegment_slices=<unavailable>, ppre_size=<unavailable>, pinfo_slices=<unavailable>, commit=<unavailable>, tld=0x00000001016cc490, os_tld=<unavailable>) at segment.c:823:42 [opt]
    frame #15: 0x000000010132ddd0 libpython3.13t.dylib`mi_segment_alloc(required=0, page_alignment=<unavailable>, req_arena_id=0, tld=<unavailable>, os_tld=<unavailable>, huge_page=0x0000000000000000) at segment.c:882:27 [opt]
    frame #16: 0x0000000101326a38 libpython3.13t.dylib`mi_segments_page_alloc [inlined] mi_segment_reclaim_or_alloc(heap=0x00000001016cac80, needed_slices=1, block_size=64, tld=0x00000001016cc490, os_tld=0x00000001016cc828) at segment.c:1489:10 [opt]
    frame #17: 0x0000000101326788 libpython3.13t.dylib`mi_segments_page_alloc(heap=0x00000001016cac80, page_kind=<unavailable>, required=64, block_size=64, tld=0x00000001016cc490, os_tld=0x00000001016cc828) at segment.c:1508:9 [opt]
    frame #18: 0x000000010132cc7c libpython3.13t.dylib`mi_page_fresh_alloc(heap=0x00000001016cac80, pq=0x00000001016cb150, block_size=64, page_alignment=<unavailable>) at page.c:284:21 [opt]
    frame #19: 0x0000000101324044 libpython3.13t.dylib`mi_find_page [inlined] mi_page_fresh(heap=0x00000001016cac80, pq=0x00000001016cb150) at page.c:305:21 [opt]
    frame #20: 0x0000000101324030 libpython3.13t.dylib`mi_find_page [inlined] mi_page_queue_find_free_ex(heap=0x00000001016cac80, pq=0x00000001016cb150, first_try=true) at page.c:782:12 [opt]
    frame #21: 0x0000000101323ff0 libpython3.13t.dylib`mi_find_page [inlined] mi_find_free_page(heap=0x00000001016cac80, size=<unavailable>) at page.c:821:10 [opt]
    frame #22: 0x0000000101323e6c libpython3.13t.dylib`mi_find_page(heap=0x00000001016cac80, size=64, huge_alignment=0) at page.c:920:12 [opt]
    frame #23: 0x0000000101316158 libpython3.13t.dylib`_mi_malloc_generic(heap=0x00000001016cac80, size=<unavailable>, zero=<unavailable>, huge_alignment=0) at page.c:946:21 [opt]
    frame #24: 0x0000000101422014 libpython3.13t.dylib`gc_alloc [inlined] _PyObject_MallocWithType(tp=<unavailable>, size=<unavailable>) at pycore_object_alloc.h:46:17 [opt]
    frame #25: 0x0000000101421ff4 libpython3.13t.dylib`gc_alloc(tp=<unavailable>, basicsize=<unavailable>, presize=0) at gc_free_threading.c:1695:17 [opt]
    frame #26: 0x0000000101421ea4 libpython3.13t.dylib`_PyObject_GC_New(tp=0x0000000101657268) at gc_free_threading.c:1716:20 [opt]
    frame #27: 0x00000001012f266c libpython3.13t.dylib`PyDict_New [inlined] new_dict(interp=0x0000000101698e80, keys=<unavailable>, values=0x0000000000000000, used=0, free_values_on_failure=0) at dictobject.c:929:14 [opt]
    frame #28: 0x00000001012f263c libpython3.13t.dylib`PyDict_New at dictobject.c:1026:12 [opt]
    frame #29: 0x0000000101386050 libpython3.13t.dylib`_PyUnicode_InitGlobalObjects [inlined] init_interned_dict(interp=0x0000000101698e80) at unicodeobject.c:284:37 [opt]
    frame #30: 0x000000010138604c libpython3.13t.dylib`_PyUnicode_InitGlobalObjects(interp=0x0000000101698e80) at unicodeobject.c:15016:9 [opt]
    frame #31: 0x00000001014521f0 libpython3.13t.dylib`pycore_interp_init [inlined] pycore_init_global_objects(interp=0x0000000101698e80) at pylifecycle.c:702:14 [opt]
    frame #32: 0x00000001014521dc libpython3.13t.dylib`pycore_interp_init(tstate=0x00000001016c9330) at pylifecycle.c:852:14 [opt]
    frame #33: 0x000000010144f89c libpython3.13t.dylib`Py_InitializeFromConfig [inlined] pyinit_config(runtime=<unavailable>, tstate_p=<unavailable>, config=0x0000000170212130) at pylifecycle.c:933:14 [opt]
    frame #34: 0x000000010144f784 libpython3.13t.dylib`Py_InitializeFromConfig [inlined] pyinit_core(runtime=<unavailable>, src_config=<unavailable>, tstate_p=<unavailable>) at pylifecycle.c:1096:18 [opt]
    frame #35: 0x000000010144f784 libpython3.13t.dylib`Py_InitializeFromConfig(config=0x00000001702123a0) at pylifecycle.c:1398:14 [opt]
    frame #36: 0x000000010144f9b0 libpython3.13t.dylib`Py_InitializeEx(install_sigs=<unavailable>) at pylifecycle.c:1436:14 [opt]
    frame #37: 0x0000000100105b2c pyo3-71e3a8c0dab6095a`pyo3::gil::prepare_freethreaded_python::_$u7b$$u7b$closure$u7d$$u7d$::h2cd40f096e08bff0((null)={closure_env#0} @ 0x00000001702125c7, (null)=0x0000000170212640) at gil.rs:69:13
    frame #38: 0x00000001000654fc pyo3-71e3a8c0dab6095a`std::sync::once::Once::call_once_force::_$u7b$$u7b$closure$u7d$$u7d$::hdab74cb30759684d(p=0x0000000170212640) at once.rs:208:40
    frame #39: 0x0000000100314024 pyo3-71e3a8c0dab6095a`std::sys::sync::once::queue::Once::call::h9dcea807fd617ccf at queue.rs:183:21 [opt]
    frame #40: 0x00000001000652f0 pyo3-71e3a8c0dab6095a`std::sync::once::Once::call_once_force::h6081e1899cf15adf(self=0x00000001004d8ea0, f={closure_env#0} @ 0x000000017021271f) at once.rs:208:9
    frame #41: 0x0000000100000b0c pyo3-71e3a8c0dab6095a`pyo3::gil::prepare_freethreaded_python::hb6ac25f766ef5eb8 at gil.rs:66:5
    frame #42: 0x0000000100000b7c pyo3-71e3a8c0dab6095a`pyo3::gil::GILGuard::acquire::hcdf69afc9ea1299e at gil.rs:174:21
    frame #43: 0x00000001001a1708 pyo3-71e3a8c0dab6095a`pyo3::marker::Python::with_gil::h089ec9aca00e522e(f={closure_env#0} @ 0x000000017021279f) at marker.rs:403:21
    frame #44: 0x0000000100216c2c pyo3-71e3a8c0dab6095a`pyo3::buffer::tests::test_bytes_buffer::h458708c8fd91dd1e at buffer.rs:849:9
    frame #45: 0x0000000100035e40 pyo3-71e3a8c0dab6095a`pyo3::buffer::tests::test_bytes_buffer::_$u7b$$u7b$closure$u7d$$u7d$::h8009c230847e71d5((null)=0x00000001702127fe) at buffer.rs:848:27

Both the debug python build and the optimized, crashing Python are built from source using pyenv.

[colesbury]

That looks unrelated to the Windows issues. The traceback makes me thing there is a bug in CPython (not just PyO3). Opened a new issue: #123022

[vstinner]

That looks unrelated to the Windows issues. The traceback makes me thing there is a bug in CPython (not just PyO3). Opened a new issue: #123022

Since the issue gh-123022 has been fixed, can we close this issue as well?