Skip to content

Race in func_get_annotations under free-threading #128714

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
hawkinsp opened this issue Jan 10, 2025 · 5 comments
Closed

Race in func_get_annotations under free-threading #128714

hawkinsp opened this issue Jan 10, 2025 · 5 comments
Labels
3.13 bugs and security fixes 3.14 new features, bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-free-threading type-bug An unexpected behavior, bug, or error

Comments

@hawkinsp
Copy link
Contributor

hawkinsp commented Jan 10, 2025

Bug report

Bug description:

With CPython 3.13 from commit 65da5db TSAN reports the following race, which looks right: the initialization of p->func_annotations is racy.

Reproducer:

import concurrent.futures
import functools
import threading

num_threads = 8

def closure(b, f):
  b.wait()
  f.__annotations__

with concurrent.futures.ThreadPoolExecutor(max_workers=num_threads) as executor:
  for _ in range(100):
    b = threading.Barrier(num_threads)
    def f(x: int) -> int:
      return x + 1
    for _ in range(num_threads):
      executor.submit(functools.partial(closure, b, f))

TSAN report (first race):

WARNING: ThreadSanitizer: data race (pid=3380651)
  Read of size 8 at 0x7f02728d6840 by thread T5:
    #0 func_get_annotations /usr/local/google/home/phawkins/p/cpython/Objects/funcobject.c:769:13 (python3.13+0x232542) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #1 getset_get /usr/local/google/home/phawkins/p/cpython/Objects/descrobject.c:193:16 (python3.13+0x1ff888) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #2 _PyObject_GenericGetAttrWithDict /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1665:19 (python3.13+0x295e11) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #3 PyObject_GenericGetAttr /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1747:12 (python3.13+0x295c62) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #4 PyObject_GetAttr /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1261:18 (python3.13+0x2945c7) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #5 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:3766:28 (python3.13+0x3ee2b0) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #6 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3dea3a) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #7 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1807:12 (python3.13+0x3dea3a)
    #8 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb65f) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #9 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x572352) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #10 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x572352)
    #11 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb2d3) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #12 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb2d3)
    #13 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb355) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #14 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e4af2) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #15 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3dea3a) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #16 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1807:12 (python3.13+0x3dea3a)
    #17 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb65f) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #18 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef62f) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #19 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef62f)
    #20 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb2d3) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #21 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb2d3)
    #22 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb355) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #23 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564a32) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #24 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bddb7) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)

  Previous write of size 8 at 0x7f02728d6840 by thread T7:
    #0 func_get_annotation_dict /usr/local/google/home/phawkins/p/cpython/Objects/funcobject.c:535:9 (python3.13+0x22f46f) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #1 func_get_annotations /usr/local/google/home/phawkins/p/cpython/Objects/funcobject.c:774:19 (python3.13+0x23256c) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #2 getset_get /usr/local/google/home/phawkins/p/cpython/Objects/descrobject.c:193:16 (python3.13+0x1ff888) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #3 _PyObject_GenericGetAttrWithDict /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1665:19 (python3.13+0x295e11) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #4 PyObject_GenericGetAttr /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1747:12 (python3.13+0x295c62) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #5 PyObject_GetAttr /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1261:18 (python3.13+0x2945c7) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #6 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:3766:28 (python3.13+0x3ee2b0) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #7 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3dea3a) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #8 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1807:12 (python3.13+0x3dea3a)
    #9 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb65f) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #10 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x572352) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #11 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x572352)
    #12 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb2d3) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #13 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb2d3)
    #14 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb355) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #15 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e4af2) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #16 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3dea3a) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #17 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1807:12 (python3.13+0x3dea3a)
    #18 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb65f) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #19 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef62f) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #20 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef62f)
    #21 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb2d3) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #22 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb2d3)
    #23 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb355) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #24 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564a32) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)
    #25 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bddb7) (BuildId: 19c569fa942016d8ac49d19fd40ccb1ddded939b)

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

@hawkinsp hawkinsp added the type-bug An unexpected behavior, bug, or error label Jan 10, 2025
@ZeroIntensity ZeroIntensity added interpreter-core (Objects, Python, Grammar, and Parser dirs) 3.13 bugs and security fixes 3.14 new features, bugs and security fixes labels Jan 10, 2025
@ZeroIntensity
Copy link
Member

cc @JelleZijlstra

@xuantengh
Copy link
Contributor

xuantengh commented Jan 19, 2025

Hi, I've submittedd a PR, and it seems funcobject.c lacks the support for free-threaded builds. Maybe we should consider audit this just like #127945, and #128002 ?

@xuantengh
Copy link
Contributor

xuantengh commented Jan 19, 2025

Besides, it seems the the range object also has data races when I run the test suites test_free_threading under FT build:

WARNING: ThreadSanitizer: data race (pid=2516929)
  Read of size 8 at 0x7f9cbfaa66a0 by thread T392:
    #0 rangeiter_next ../Objects/rangeobject.c:819 (python+0x21713e) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #1 zip_next ../Python/bltinmodule.c:3111 (python+0x313553) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #2 builtin_next ../Python/bltinmodule.c:1651 (python+0x3155bb) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #3 _PyEval_EvalFrameDefault ../Python/generated_cases.c.h:1510 (python+0x32d5df) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #4 _PyEval_EvalFrame ../Include/internal/pycore_ceval.h:116 (python+0x358df3) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #5 _PyEval_Vector ../Python/ceval.c:1913 (python+0x35904f) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #6 _PyFunction_Vectorcall ../Objects/call.c:413 (python+0x165e43) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #7 _PyObject_VectorcallTstate ../Include/internal/pycore_call.h:167 (python+0x16a189) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #8 method_vectorcall ../Objects/classobject.c:72 (python+0x16a477) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #9 _PyVectorcall_Call ../Objects/call.c:273 (python+0x167f38) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #10 _PyObject_Call ../Objects/call.c:348 (python+0x168362) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #11 PyObject_Call ../Objects/call.c:373 (python+0x1683c2) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #12 thread_run ../Modules/_threadmodule.c:354 (python+0x4bcd17) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #13 pythread_wrapper ../Python/thread_pthread.h:242 (python+0x41af14) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)

  Previous write of size 8 at 0x7f9cbfaa66a0 by thread T394:
    #0 rangeiter_next ../Objects/rangeobject.c:822 (python+0x217198) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #1 zip_next ../Python/bltinmodule.c:3111 (python+0x313553) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #2 builtin_next ../Python/bltinmodule.c:1651 (python+0x3155bb) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #3 _PyEval_EvalFrameDefault ../Python/generated_cases.c.h:1510 (python+0x32d5df) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #4 _PyEval_EvalFrame ../Include/internal/pycore_ceval.h:116 (python+0x358df3) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #5 _PyEval_Vector ../Python/ceval.c:1913 (python+0x35904f) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #6 _PyFunction_Vectorcall ../Objects/call.c:413 (python+0x165e43) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #7 _PyObject_VectorcallTstate ../Include/internal/pycore_call.h:167 (python+0x16a189) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #8 method_vectorcall ../Objects/classobject.c:72 (python+0x16a477) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #9 _PyVectorcall_Call ../Objects/call.c:273 (python+0x167f38) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #10 _PyObject_Call ../Objects/call.c:348 (python+0x168362) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #11 PyObject_Call ../Objects/call.c:373 (python+0x1683c2) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #12 thread_run ../Modules/_threadmodule.c:354 (python+0x4bcd17) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)
    #13 pythread_wrapper ../Python/thread_pthread.h:242 (python+0x41af14) (BuildId: 3d91448656453eabf7067b9e5af5b37997c21fc8)

@hawkinsp
Copy link
Contributor Author

Here's a somewhat similar looking race on type_get_annotations:

WARNING: ThreadSanitizer: data race (pid=355399)
  Read of size 4 at 0x7e07ad0c2190 by thread T65 (mutexes: read M0):
    #0 PyType_Modified /__w/jax/jax/cpython/Objects/typeobject.c:1059:15 (python3.13+0x305823) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80)
    #1 type_get_annotations /__w/jax/jax/cpython/Objects/typeobject.c:1790:17 (python3.13+0x305823)
    #2 getset_get /__w/jax/jax/cpython/Objects/descrobject.c:193:16 (python3.13+0x1ff5e8) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80)
    #3 _Py_type_getattro_impl /__w/jax/jax/cpython/Objects/typeobject.c:5411:19 (python3.13+0x2e92cd) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80)
    #4 PyObject_GetOptionalAttr /__w/jax/jax/cpython/Objects/object.c:1307:19 (python3.13+0x292eb1) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80)
    #5 builtin_getattr /__w/jax/jax/cpython/Python/bltinmodule.c:1200:13 (python3.13+0x3d8d79) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80)
    #6 cfunction_vectorcall_FASTCALL /__w/jax/jax/cpython/Objects/methodobject.c:425:24 (python3.13+0x289b7b) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80)
    #7 _PyObject_VectorcallTstate /__w/jax/jax/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ead4a) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80)

...

Previous atomic write of size 4 at 0x7e07ad0c2190 by thread T67 (mutexes: read M0):
    #0 _Py_atomic_store_uint32_relaxed /__w/jax/jax/cpython/./Include/cpython/pyatomic_gcc.h:461:3 (python3.13+0x2e5090) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80)
    #1 set_version_unlocked /__w/jax/jax/cpython/Objects/typeobject.c:978:5 (python3.13+0x2e5090)
    #2 type_modified_unlocked /__w/jax/jax/cpython/Objects/typeobject.c:1047:5 (python3.13+0x2e5090)
    #3 PyType_Modified /__w/jax/jax/cpython/Objects/typeobject.c:1064:5 (python3.13+0x3058d4) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80)
    #4 type_get_annotations /__w/jax/jax/cpython/Objects/typeobject.c:1790:17 (python3.13+0x3058d4)
    #5 getset_get /__w/jax/jax/cpython/Objects/descrobject.c:193:16 (python3.13+0x1ff5e8) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80)
    #6 _Py_type_getattro_impl /__w/jax/jax/cpython/Objects/typeobject.c:5411:19 (python3.13+0x2e92cd) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80)
    #7 PyObject_GetOptionalAttr /__w/jax/jax/cpython/Objects/object.c:1307:19 (python3.13+0x292eb1) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80)
 
...

I figured it might be simplest just to add this here, but I can open a new issue if you like!

hawkinsp added a commit to hawkinsp/jax that referenced this issue Jan 31, 2025

Verified

This commit was signed with the committer’s verified signature.
lunny Lunny Xiao
@kumaraditya303
Copy link
Contributor

I figured it might be simplest just to add this here, but I can open a new issue if you like!

Please create a new issue for it, the getters and setters of typeobject are missing critical section which would have to be added independent of function object fixes.

kumaraditya303 pushed a commit that referenced this issue Feb 6, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
…s__` and `__type_params__` in free-threading build (#129016)
xuantengh added a commit to xuantengh/cpython that referenced this issue Feb 6, 2025
…tations__` and `__type_params__` in free-threading build (python#129016)

(cherry picked from commit 55f17b7)
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this issue Feb 7, 2025
…tations__` and `__type_params__` in free-threading build (python#129016)
cmaloney pushed a commit to cmaloney/cpython that referenced this issue Feb 8, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
…tations__` and `__type_params__` in free-threading build (python#129016)
terryjreedy pushed a commit that referenced this issue Feb 24, 2025
…otations__` and `__type_params__` in free-threading build (GH-129016) (#129729)

* gh-128714: Fix function object races in `__annotate__`, `__annotations__` and `__type_params__` in  free-threading build (#129016)

(cherry picked from commit 55f17b7)

---------

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.13 bugs and security fixes 3.14 new features, bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-free-threading type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

5 participants