Skip to content
Closed
24 changes: 24 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -472,6 +472,29 @@ jobs:
- name: Tests
run: xvfb-run make test

build_tsan_free_threading:
name: 'Thread sanitizer (free-threading)'
runs-on: ubuntu-20.04
timeout-minutes: 60
needs: check_source
if: needs.check_source.outputs.run_tests == 'true'
steps:
- uses: actions/checkout@v4
- name: Install Dependencies
run: sudo ./.github/workflows/posix-deps-apt.sh
- name: Set up GCC-10 for ASAN
uses: egor-tensin/setup-gcc@v1
with:
version: 11
Comment on lines +485 to +488
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The version says "11" but the step name says "GCC-10"

- name: Configure CPython
run: ./configure --disable-gil --with-thread-sanitizer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want --with-pydebug?

- name: Build CPython
run: make -j4
- name: Display build info
run: make pythoninfo
- name: Tests
run: ./python -m test --pgo -j4 # Reduce test scope
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'll want a configuration specifically for tsan. Maybe a --tsan option? We'll only want to run tests that actually use threading.

Brett had a list of tests that fail with PYTHON_GIL=0 -- those are probably a good starting point:

swtaarrs@1fe9165#diff-590c02490d066378045dd48d62e86dc85d3fb4ad934b6a2b2b0b1112e59eaefb


# CIFuzz job based on https://google.github.io/oss-fuzz/getting-started/continuous-integration/
cifuzz:
name: CIFuzz
Expand Down Expand Up @@ -561,6 +584,7 @@ jobs:
build_windows,
build_windows_free_threading,
build_asan,
build_tsan_free_threading,
'
|| ''
}}
Expand Down
1 change: 1 addition & 0 deletions Objects/obmalloc.c
Original file line number Diff line number Diff line change
Expand Up @@ -446,6 +446,7 @@ _PyMem_ArenaFree(void *Py_UNUSED(ctx), void *ptr,
/***************************/

static int
_Py_NO_SANITIZE_THREAD
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've had a better experience using suppression lists rather than marking functions as _Py_NO_SANITIZE_THREAD:

  1. Add a file supressions.txt somwhere (maybe Tools/tsan/supressions.txt?)
  2. Set the environment variable TSAN_OPTIONS="suppressions=<path_to_supressions.txt>"

See https://github.com/google/sanitizers/wiki/ThreadSanitizerSuppressions

set_default_allocator_unlocked(PyMemAllocatorDomain domain, int debug,
PyMemAllocatorEx *old_alloc)
{
Expand Down
7 changes: 7 additions & 0 deletions configure

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -3264,6 +3264,12 @@ AC_MSG_RESULT([no])
with_tsan="no"
])

if test "$with_tsan" = "yes"
then
AC_DEFINE([MI_TSAN], [1],
[Define MI_TSAN for thread sanitizer])
fi

# Set info about shared libraries.
AC_SUBST([SHLIB_SUFFIX])
AC_SUBST([LDSHARED])
Expand Down
3 changes: 3 additions & 0 deletions pyconfig.h.in
Original file line number Diff line number Diff line change
Expand Up @@ -1588,6 +1588,9 @@
<sysmacros.h>. */
#undef MAJOR_IN_SYSMACROS

/* Define MI_TSAN for thread sanitizer */
#undef MI_TSAN

/* Define if mvwdelch in curses.h is an expression. */
#undef MVWDELCH_IS_EXPRESSION

Expand Down