Skip to content

Commit 01cb96a

Browse files
cfbolzcfbolz
committed
Merge branch 'main' into pythongh-82378-fix-tracebacklimit-in-pyrepl
2 parents 1d4f321 + bd3d31f commit 01cb96a

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+1112
-680
lines changed

‎.github/workflows/build.yml

+13-118
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,5 @@
11
name: Tests
22

3-
# gh-84728: "paths-ignore" is not used to skip documentation-only PRs, because
4-
# it prevents to mark a job as mandatory. A PR cannot be merged if a job is
5-
# mandatory but not scheduled because of "paths-ignore".
63
on:
74
workflow_dispatch:
85
push:
@@ -23,121 +20,19 @@ concurrency:
2320

2421
jobs:
2522
check_source:
26-
name: 'Check for source changes'
27-
runs-on: ubuntu-latest
28-
timeout-minutes: 10
29-
outputs:
30-
# Some of the referenced steps set outputs conditionally and there may be
31-
# cases when referencing them evaluates to empty strings. It is nice to
32-
# work with proper booleans so they have to be evaluated through JSON
33-
# conversion in the expressions. However, empty strings used like that
34-
# may trigger all sorts of undefined and hard-to-debug behaviors in
35-
# GitHub Actions CI/CD. To help with this, all of the outputs set here
36-
# that are meant to be used as boolean flags (and not arbitrary strings),
37-
# MUST have fallbacks with default values set. A common pattern would be
38-
# to add ` || false` to all such expressions here, in the output
39-
# definitions. They can then later be safely used through the following
40-
# idiom in job conditionals and other expressions. Here's some examples:
41-
#
42-
# if: fromJSON(needs.check_source.outputs.run-docs)
43-
#
44-
# ${{
45-
# fromJSON(needs.check_source.outputs.run_tests)
46-
# && 'truthy-branch'
47-
# || 'falsy-branch'
48-
# }}
49-
#
50-
run-docs: ${{ steps.docs-changes.outputs.run-docs || false }}
51-
run-win-msi: ${{ steps.win-msi-changes.outputs.run-win-msi || false }}
52-
run_tests: ${{ steps.check.outputs.run_tests || false }}
53-
run_hypothesis: ${{ steps.check.outputs.run_hypothesis || false }}
54-
run_cifuzz: ${{ steps.check.outputs.run_cifuzz || false }}
55-
config_hash: ${{ steps.config_hash.outputs.hash }} # str
56-
steps:
57-
- uses: actions/checkout@v4
58-
- name: Check for source changes
59-
id: check
60-
run: |
61-
if [ -z "$GITHUB_BASE_REF" ]; then
62-
echo "run_tests=true" >> $GITHUB_OUTPUT
63-
else
64-
git fetch origin $GITHUB_BASE_REF --depth=1
65-
# git diff "origin/$GITHUB_BASE_REF..." (3 dots) may be more
66-
# reliable than git diff "origin/$GITHUB_BASE_REF.." (2 dots),
67-
# but it requires to download more commits (this job uses
68-
# "git fetch --depth=1").
69-
#
70-
# git diff "origin/$GITHUB_BASE_REF..." (3 dots) works with Git
71-
# 2.26, but Git 2.28 is stricter and fails with "no merge base".
72-
#
73-
# git diff "origin/$GITHUB_BASE_REF.." (2 dots) should be enough on
74-
# GitHub, since GitHub starts by merging origin/$GITHUB_BASE_REF
75-
# into the PR branch anyway.
76-
#
77-
# https://github.com/python/core-workflow/issues/373
78-
git diff --name-only origin/$GITHUB_BASE_REF.. | grep -qvE '(\.rst$|^Doc|^Misc|^\.pre-commit-config\.yaml$|\.ruff\.toml$|\.md$|mypy\.ini$)' && echo "run_tests=true" >> $GITHUB_OUTPUT || true
79-
fi
80-
81-
# Check if we should run hypothesis tests
82-
GIT_BRANCH=${GITHUB_BASE_REF:-${GITHUB_REF#refs/heads/}}
83-
echo $GIT_BRANCH
84-
if $(echo "$GIT_BRANCH" | grep -q -w '3\.\(8\|9\|10\|11\)'); then
85-
echo "Branch too old for hypothesis tests"
86-
echo "run_hypothesis=false" >> $GITHUB_OUTPUT
87-
else
88-
echo "Run hypothesis tests"
89-
echo "run_hypothesis=true" >> $GITHUB_OUTPUT
90-
fi
91-
92-
# oss-fuzz maintains a configuration for fuzzing the main branch of
93-
# CPython, so CIFuzz should be run only for code that is likely to be
94-
# merged into the main branch; compatibility with older branches may
95-
# be broken.
96-
FUZZ_RELEVANT_FILES='(\.c$|\.h$|\.cpp$|^configure$|^\.github/workflows/build\.yml$|^Modules/_xxtestfuzz)'
97-
if [ "$GITHUB_BASE_REF" = "main" ] && [ "$(git diff --name-only origin/$GITHUB_BASE_REF.. | grep -qE $FUZZ_RELEVANT_FILES; echo $?)" -eq 0 ]; then
98-
# The tests are pretty slow so they are executed only for PRs
99-
# changing relevant files.
100-
echo "Run CIFuzz tests"
101-
echo "run_cifuzz=true" >> $GITHUB_OUTPUT
102-
else
103-
echo "Branch too old for CIFuzz tests; or no C files were changed"
104-
echo "run_cifuzz=false" >> $GITHUB_OUTPUT
105-
fi
106-
- name: Compute hash for config cache key
107-
id: config_hash
108-
run: |
109-
echo "hash=${{ hashFiles('configure', 'configure.ac', '.github/workflows/build.yml') }}" >> $GITHUB_OUTPUT
110-
- name: Get a list of the changed documentation-related files
111-
if: github.event_name == 'pull_request'
112-
id: changed-docs-files
113-
uses: Ana06/get-changed-files@v2.3.0
114-
with:
115-
filter: |
116-
Doc/**
117-
Misc/**
118-
.github/workflows/reusable-docs.yml
119-
format: csv # works for paths with spaces
120-
- name: Check for docs changes
121-
if: >-
122-
github.event_name == 'pull_request'
123-
&& steps.changed-docs-files.outputs.added_modified_renamed != ''
124-
id: docs-changes
125-
run: |
126-
echo "run-docs=true" >> "${GITHUB_OUTPUT}"
127-
- name: Get a list of the MSI installer-related files
128-
id: changed-win-msi-files
129-
uses: Ana06/get-changed-files@v2.3.0
130-
with:
131-
filter: |
132-
Tools/msi/**
133-
.github/workflows/reusable-windows-msi.yml
134-
format: csv # works for paths with spaces
135-
- name: Check for changes in MSI installer-related files
136-
if: >-
137-
steps.changed-win-msi-files.outputs.added_modified_renamed != ''
138-
id: win-msi-changes
139-
run: |
140-
echo "run-win-msi=true" >> "${GITHUB_OUTPUT}"
23+
name: Change detection
24+
# To use boolean outputs from this job, parse them as JSON.
25+
# Here's some examples:
26+
#
27+
# if: fromJSON(needs.check_source.outputs.run-docs)
28+
#
29+
# ${{
30+
# fromJSON(needs.check_source.outputs.run_tests)
31+
# && 'truthy-branch'
32+
# || 'falsy-branch'
33+
# }}
34+
#
35+
uses: ./.github/workflows/reusable-change-detection.yml
14136

14237
check-docs:
14338
name: Docs

‎.github/workflows/reusable-change-detection.yml

+150
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
---
2+
3+
name: Change detection
4+
5+
on: # yamllint disable-line rule:truthy
6+
workflow_call:
7+
outputs:
8+
# Some of the referenced steps set outputs conditionally and there may be
9+
# cases when referencing them evaluates to empty strings. It is nice to
10+
# work with proper booleans so they have to be evaluated through JSON
11+
# conversion in the expressions. However, empty strings used like that
12+
# may trigger all sorts of undefined and hard-to-debug behaviors in
13+
# GitHub Actions CI/CD. To help with this, all of the outputs set here
14+
# that are meant to be used as boolean flags (and not arbitrary strings),
15+
# MUST have fallbacks with default values set. A common pattern would be
16+
# to add ` || false` to all such expressions here, in the output
17+
# definitions. They can then later be safely used through the following
18+
# idiom in job conditionals and other expressions. Here's some examples:
19+
#
20+
# if: fromJSON(needs.change-detection.outputs.run-docs)
21+
#
22+
# ${{
23+
# fromJSON(needs.change-detection.outputs.run-tests)
24+
# && 'truthy-branch'
25+
# || 'falsy-branch'
26+
# }}
27+
#
28+
config_hash:
29+
description: Config hash value for use in cache keys
30+
value: ${{ jobs.compute-changes.outputs.config-hash }} # str
31+
run-docs:
32+
description: Whether to build the docs
33+
value: ${{ jobs.compute-changes.outputs.run-docs || false }} # bool
34+
run_tests:
35+
description: Whether to run the regular tests
36+
value: ${{ jobs.compute-changes.outputs.run-tests || false }} # bool
37+
run-win-msi:
38+
description: Whether to run the MSI installer smoke tests
39+
value: >- # bool
40+
${{ jobs.compute-changes.outputs.run-win-msi || false }}
41+
run_hypothesis:
42+
description: Whether to run the Hypothesis tests
43+
value: >- # bool
44+
${{ jobs.compute-changes.outputs.run-hypothesis || false }}
45+
run_cifuzz:
46+
description: Whether to run the CIFuzz job
47+
value: >- # bool
48+
${{ jobs.compute-changes.outputs.run-cifuzz || false }}
49+
50+
jobs:
51+
compute-changes:
52+
name: Compute changed files
53+
runs-on: ubuntu-latest
54+
timeout-minutes: 10
55+
outputs:
56+
config-hash: ${{ steps.config-hash.outputs.hash }}
57+
run-cifuzz: ${{ steps.check.outputs.run-cifuzz }}
58+
run-docs: ${{ steps.docs-changes.outputs.run-docs }}
59+
run-hypothesis: ${{ steps.check.outputs.run-hypothesis }}
60+
run-tests: ${{ steps.check.outputs.run-tests }}
61+
run-win-msi: ${{ steps.win-msi-changes.outputs.run-win-msi }}
62+
steps:
63+
- run: >-
64+
echo '${{ github.event_name }}'
65+
- uses: actions/checkout@v4
66+
- name: Check for source changes
67+
id: check
68+
run: |
69+
if [ -z "$GITHUB_BASE_REF" ]; then
70+
echo "run-tests=true" >> $GITHUB_OUTPUT
71+
else
72+
git fetch origin $GITHUB_BASE_REF --depth=1
73+
# git diff "origin/$GITHUB_BASE_REF..." (3 dots) may be more
74+
# reliable than git diff "origin/$GITHUB_BASE_REF.." (2 dots),
75+
# but it requires to download more commits (this job uses
76+
# "git fetch --depth=1").
77+
#
78+
# git diff "origin/$GITHUB_BASE_REF..." (3 dots) works with Git
79+
# 2.26, but Git 2.28 is stricter and fails with "no merge base".
80+
#
81+
# git diff "origin/$GITHUB_BASE_REF.." (2 dots) should be enough on
82+
# GitHub, since GitHub starts by merging origin/$GITHUB_BASE_REF
83+
# into the PR branch anyway.
84+
#
85+
# https://github.com/python/core-workflow/issues/373
86+
git diff --name-only origin/$GITHUB_BASE_REF.. | grep -qvE '(\.rst$|^Doc|^Misc|^\.pre-commit-config\.yaml$|\.ruff\.toml$|\.md$|mypy\.ini$)' && echo "run-tests=true" >> $GITHUB_OUTPUT || true
87+
fi
88+
89+
# Check if we should run hypothesis tests
90+
GIT_BRANCH=${GITHUB_BASE_REF:-${GITHUB_REF#refs/heads/}}
91+
echo $GIT_BRANCH
92+
if $(echo "$GIT_BRANCH" | grep -q -w '3\.\(8\|9\|10\|11\)'); then
93+
echo "Branch too old for hypothesis tests"
94+
echo "run-hypothesis=false" >> $GITHUB_OUTPUT
95+
else
96+
echo "Run hypothesis tests"
97+
echo "run-hypothesis=true" >> $GITHUB_OUTPUT
98+
fi
99+
100+
# oss-fuzz maintains a configuration for fuzzing the main branch of
101+
# CPython, so CIFuzz should be run only for code that is likely to be
102+
# merged into the main branch; compatibility with older branches may
103+
# be broken.
104+
FUZZ_RELEVANT_FILES='(\.c$|\.h$|\.cpp$|^configure$|^\.github/workflows/build\.yml$|^Modules/_xxtestfuzz)'
105+
if [ "$GITHUB_BASE_REF" = "main" ] && [ "$(git diff --name-only origin/$GITHUB_BASE_REF.. | grep -qE $FUZZ_RELEVANT_FILES; echo $?)" -eq 0 ]; then
106+
# The tests are pretty slow so they are executed only for PRs
107+
# changing relevant files.
108+
echo "Run CIFuzz tests"
109+
echo "run-cifuzz=true" >> $GITHUB_OUTPUT
110+
else
111+
echo "Branch too old for CIFuzz tests; or no C files were changed"
112+
echo "run-cifuzz=false" >> $GITHUB_OUTPUT
113+
fi
114+
- name: Compute hash for config cache key
115+
id: config-hash
116+
run: |
117+
echo "hash=${{ hashFiles('configure', 'configure.ac', '.github/workflows/build.yml') }}" >> $GITHUB_OUTPUT
118+
- name: Get a list of the changed documentation-related files
119+
if: github.event_name == 'pull_request'
120+
id: changed-docs-files
121+
uses: Ana06/get-changed-files@v2.3.0
122+
with:
123+
filter: |
124+
Doc/**
125+
Misc/**
126+
.github/workflows/reusable-docs.yml
127+
format: csv # works for paths with spaces
128+
- name: Check for docs changes
129+
if: >-
130+
github.event_name == 'pull_request'
131+
&& steps.changed-docs-files.outputs.added_modified_renamed != ''
132+
id: docs-changes
133+
run: |
134+
echo "run-docs=true" >> "${GITHUB_OUTPUT}"
135+
- name: Get a list of the MSI installer-related files
136+
id: changed-win-msi-files
137+
uses: Ana06/get-changed-files@v2.3.0
138+
with:
139+
filter: |
140+
Tools/msi/**
141+
.github/workflows/reusable-windows-msi.yml
142+
format: csv # works for paths with spaces
143+
- name: Check for changes in MSI installer-related files
144+
if: >-
145+
steps.changed-win-msi-files.outputs.added_modified_renamed != ''
146+
id: win-msi-changes
147+
run: |
148+
echo "run-win-msi=true" >> "${GITHUB_OUTPUT}"
149+
150+
...

‎.github/workflows/reusable-tsan.yml

+5-5
Original file line numberDiff line numberDiff line change
@@ -36,11 +36,11 @@ jobs:
3636
# Install clang-18
3737
wget https://apt.llvm.org/llvm.sh
3838
chmod +x llvm.sh
39-
sudo ./llvm.sh 18
40-
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-18 100
41-
sudo update-alternatives --set clang /usr/bin/clang-18
42-
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-18 100
43-
sudo update-alternatives --set clang++ /usr/bin/clang++-18
39+
sudo ./llvm.sh 17 # gh-121946: llvm-18 package is temporarily broken
40+
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-17 100
41+
sudo update-alternatives --set clang /usr/bin/clang-17
42+
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-17 100
43+
sudo update-alternatives --set clang++ /usr/bin/clang++-17
4444
# Reduce ASLR to avoid TSAN crashing
4545
sudo sysctl -w vm.mmap_rnd_bits=28
4646
- name: TSAN Option Setup

‎.github/workflows/reusable-ubuntu.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ jobs:
7575
${{ fromJSON(inputs.free-threading) && '--disable-gil' || '' }}
7676
- name: Build CPython out-of-tree
7777
working-directory: ${{ env.CPYTHON_BUILDDIR }}
78-
run: make -j4 &> compiler_output.txt
78+
run: set -o pipefail; make -j4 2>&1 | tee compiler_output.txt
7979
- name: Display build info
8080
working-directory: ${{ env.CPYTHON_BUILDDIR }}
8181
run: make pythoninfo

‎Android/android.py

+19-6
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
#!/usr/bin/env python3
22

33
import argparse
4+
from glob import glob
45
import os
56
import re
67
import shutil
@@ -16,16 +17,21 @@
1617
CROSS_BUILD_DIR = CHECKOUT / "cross-build"
1718

1819

19-
def delete_if_exists(path):
20-
if path.exists():
20+
def delete_glob(pattern):
21+
# Path.glob doesn't accept non-relative patterns.
22+
for path in glob(str(pattern)):
23+
path = Path(path)
2124
print(f"Deleting {path} ...")
22-
shutil.rmtree(path)
25+
if path.is_dir() and not path.is_symlink():
26+
shutil.rmtree(path)
27+
else:
28+
path.unlink()
2329

2430

2531
def subdir(name, *, clean=None):
2632
path = CROSS_BUILD_DIR / name
2733
if clean:
28-
delete_if_exists(path)
34+
delete_glob(path)
2935
if not path.exists():
3036
if clean is None:
3137
sys.exit(
@@ -150,10 +156,17 @@ def configure_host_python(context):
150156

151157

152158
def make_host_python(context):
159+
# The CFLAGS and LDFLAGS set in android-env include the prefix dir, so
160+
# delete any previously-installed Python libs and include files to prevent
161+
# them being used during the build.
153162
host_dir = subdir(context.host)
163+
prefix_dir = host_dir / "prefix"
164+
delete_glob(f"{prefix_dir}/include/python*")
165+
delete_glob(f"{prefix_dir}/lib/libpython*")
166+
154167
os.chdir(host_dir / "build")
155168
run(["make", "-j", str(os.cpu_count())], host=context.host)
156-
run(["make", "install", f"prefix={host_dir}/prefix"], host=context.host)
169+
run(["make", "install", f"prefix={prefix_dir}"], host=context.host)
157170

158171

159172
def build_all(context):
@@ -164,7 +177,7 @@ def build_all(context):
164177

165178

166179
def clean_all(context):
167-
delete_if_exists(CROSS_BUILD_DIR)
180+
delete_glob(CROSS_BUILD_DIR)
168181

169182

170183
# To avoid distributing compiled artifacts without corresponding source code,

0 commit comments

Comments
 (0)