diff --git a/contributor-docs/python-tips.md b/contributor-docs/python-tips.md index 37c0682e8d23..b582dfbadd9f 100644 --- a/contributor-docs/python-tips.md +++ b/contributor-docs/python-tips.md @@ -265,7 +265,17 @@ Execute the following code for running tests using tox: ### Running Tests Using gradle -Integration tests suites on Jenkins are configured in groovy files that launch certain gradle tasks ([example](https://github.com/apache/beam/blob/0fd6a044df5b9f26d567e0f9a619a665a0f4043b/.test-infra/jenkins/job_PostCommit_Python.groovy#L43)). You could launch test suites locally by executing the gradle targets directly (for example: `./gradlew :sdks:python:test-suites:dataflow:py39:postCommitPy39`). This option may only be available to committers, as by default the test suites are configured to use the [`apache-beam-testing`](https://github.com/apache/beam/blob/0fd6a044df5b9f26d567e0f9a619a665a0f4043b/sdks/python/scripts/run_integration_test.sh#L70) project. +Integration tests suites on Jenkins are configured in groovy files that launch certain gradle tasks ([example](https://github.com/apache/beam/blob/0fd6a044df5b9f26d567e0f9a619a665a0f4043b/.test-infra/jenkins/job_PostCommit_Python.groovy#L43)). You could launch test suites locally by executing the gradle targets directly (for example: `./gradlew :sdks:python:test-suites:dataflow:py39:postCommitPy39`). This option may only be available to committers, as by default the test suites are configured to use the [`apache-beam-testing`](https://github.com/apache/beam/blob/0fd6a044df5b9f26d567e0f9a619a665a0f4043b/sdks/python/scripts/run_integration_test.sh#L70) project. + +### Environment Variables for Test Stability + +The following environment variables can be used to improve test stability in CI environments: + +**Test execution settings:** +- `PYTEST_XDIST_WORKER_COUNT=1` - Force sequential test execution +- `PYTHONHASHSEED=0` - Ensure deterministic hash behavior +- `OMP_NUM_THREADS=1` - Limit OpenMP threads +- `OPENBLAS_NUM_THREADS=1` - Limit OpenBLAS threads To run only a subset of tests using this approach, you could adjust the test label in the test (such as [it_postcommit](https://github.com/apache/beam/blob/25e6008e8919c2f31eaebae2662b44e02f9f37a1/sdks/python/apache_beam/io/gcp/pubsub_integration_test.py#L211)) and the [selector](https://github.com/apache/beam/blob/25e6008e8919c2f31eaebae2662b44e02f9f37a1/sdks/python/test-suites/dataflow/common.gradle#L117) where the test suite is defined. diff --git a/sdks/python/scripts/run_pytest.sh b/sdks/python/scripts/run_pytest.sh index e016907cc1a8..ec1cc2547fef 100755 --- a/sdks/python/scripts/run_pytest.sh +++ b/sdks/python/scripts/run_pytest.sh @@ -152,4 +152,4 @@ if [[ $status1 != 0 && $status1 != 5 ]]; then fi if [[ $status2 != 0 && $status2 != 5 ]]; then exit $status2 -fi \ No newline at end of file +fi diff --git a/sdks/python/test-suites/tox/common.gradle b/sdks/python/test-suites/tox/common.gradle index ac5dc57d8a55..9f79fd6ecb71 100644 --- a/sdks/python/test-suites/tox/common.gradle +++ b/sdks/python/test-suites/tox/common.gradle @@ -34,10 +34,9 @@ test.dependsOn "testPy${pythonVersionSuffix}Dill" // toxTask "testPy${pythonVersionSuffix}Dask", "py${pythonVersionSuffix}-dask", "${posargs}" // test.dependsOn "testPy${pythonVersionSuffix}Dask" +// Since codecoverage reports will always be generated for py39, +// all tests will be exercised. project.tasks.register("preCommitPy${pythonVersionSuffix}") { - // Since codecoverage reports will always be generated for py39, - // all tests will be exercised. - // dependsOn = ["testPy${pythonVersionSuffix}Cloud", "testPython${pythonVersionSuffix}"] dependsOn = ["testPy${pythonVersionSuffix}Cloud", "testPython${pythonVersionSuffix}"] }