gh-106584: Fix exit code for unittest in Python 3.12 #106588

EliseevEgor · 2023-07-10T11:43:23Z

Increase the number of running tests only if a test hasn't been skipped.

Issue: Return the correct exit code if all tests have been skipped in unittest #106584

bedevere-bot · 2023-07-10T11:43:28Z

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

cpython-cla-bot · 2023-07-10T11:44:11Z

All commit authors signed the Contributor License Agreement.

bedevere-bot · 2023-07-10T11:54:55Z

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

bedevere-bot · 2023-07-10T12:48:10Z

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

Increase the number of running tests only if it hasn't been skipped.

bedevere-bot · 2023-07-10T12:51:03Z

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

sobolevn

LGTM, but please add a NEWS entry!

Misc/NEWS.d/next/Library/2023-07-11-08-56-40.gh-issue-106584.g-SBtC.rst

…-SBtC.rst

Lib/unittest/case.py

bedevere-bot · 2023-07-11T09:54:39Z

🤖 New build scheduled with the buildbot fleet by @sunmy2019 for commit 9779a2b 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>

sobolevn

No need to specify the version info in the NEWS.

Misc/NEWS.d/next/Library/2023-07-11-08-56-40.gh-issue-106584.g-SBtC.rst

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

bedevere-bot · 2023-07-11T17:54:13Z

🤖 New build scheduled with the buildbot fleet by @sunmy2019 for commit 6313c98 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

sunmy2019 · 2023-07-13T07:49:01Z

We got some failed tests. Most are unrelated, but we might need to rule them out.

…ss_db_feature() test on Python 3.12.1+. Python 3.12.1+ no longer includes skipped tests in the number of running tests. Check out: python/cpython#110890 (comment) python/cpython#106588

…ass_unless_db_feature() test on Python 3.12.1+. Python 3.12.1+ no longer includes skipped tests in the number of running tests. Check out: python/cpython#110890 (comment) python/cpython#106588 Backport of 20b7aac from main

exarkun · 2023-12-18T15:42:29Z

It doesn't seem like this ought to have been backported. It includes an incompatible change in the long-established protocol between a test runner and a test results object. One consequence is that it has broken trial's multiprocess test runner (twisted/twisted#12052).

In Python 3.12.1 python/cpython#106588 was backported which changed the execution behavior of the unittest runner. After python/cpython#106588 startTest() is no longer being called if a test is skipped. This causes knock-on effects in testtools because the test result subclasses were assuming that startTest() was always called when stopTest() was called. To handle this change in behavior when running with Python 3.12.1 this commit adds a check to only deal with tags if they exist (when startTest() is run).

yilei · 2024-01-11T23:49:59Z

FYI-- This also broke absl.testing where it tracks test runtime with startTest and stopTest: https://github.com/abseil/abseil-py/blob/2af184bfcdaa6156a5a098923a9561bc4b7d988c/absl/testing/xml_reporter.py#L352-L365

We should be able to workaround it, but just wanted to point this out. It seems strange for stopTest to be called without a startTest.

…ipped. `startTest` may not be called in this case, after the change in python/cpython#106588. PiperOrigin-RevId: 597675639

…ipped. `startTest` may not be called in this case, after the change in python/cpython#106588. PiperOrigin-RevId: 597840713

In Python 3.12.1 python/cpython#106588 was backported which changed the execution behavior of the unittest runner. After python/cpython#106588 startTest() is no longer being called if a test is skipped. This causes knock-on effects in testtools because the test result subclasses were assuming that startTest() was always called when stopTest() was called. To handle this change in behavior when running with Python 3.12.1 this commit adds a check to only deal with tags if they exist (when startTest() is run).

* Add Python 3.12 builds The migrator seems confused by another package from a split feedstock that also produces non-Python packages that qiskit-aer depends on not having Python 3.12 builds yet, so this is a manual migration. Explicitly exclude TestNoiseTransformer tests because they require cvxpy. Previously they were skipped by unittest when cvxpy was not installed but test skipping was broken by Python 3.12.1 (see testing-cabal/testtools#371 and python/cpython#106588), so we had to explicitly skip the tests with the test runner.

…ython#106588)" This reverts commit 8fc0713.

…12 (#106588)" (GH-114470) This reverts commit 8fc0713.

…in Python 3.12 (pythonGH-106588)" (pythonGH-114470) This reverts commit 8fc0713. (cherry picked from commit ecabff9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…thon 3.12 (GH-106588)" (GH-114470) (GH-114994) This reverts commit 8fc0713. (cherry picked from commit ecabff9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…in Python 3.12 (python#106588)" (pythonGH-114470) This reverts commit 8fc0713.

…6588) Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com> Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

bedevere-bot added the awaiting review label Jul 10, 2023

bedevere-bot mentioned this pull request Jul 10, 2023

Return the correct exit code if all tests have been skipped in unittest #106584

Closed

EliseevEgor force-pushed the egoreliseev/Issue-106584_move_start_test branch from 2b0b119 to 9efdf58 Compare July 10, 2023 12:48

pythonGH-106584: Fix exit code for unittest in Python 3.12

6e1be71

Increase the number of running tests only if it hasn't been skipped.

EliseevEgor force-pushed the egoreliseev/Issue-106584_move_start_test branch from 9efdf58 to 6e1be71 Compare July 10, 2023 12:51

sobolevn approved these changes Jul 11, 2023

View reviewed changes

bedevere-bot added awaiting core review and removed awaiting review labels Jul 11, 2023

📜🤖 Added by blurb_it.

ee3af81

sunmy2019 reviewed Jul 11, 2023

View reviewed changes

Misc/NEWS.d/next/Library/2023-07-11-08-56-40.gh-issue-106584.g-SBtC.rst Outdated Show resolved Hide resolved

Update Misc/NEWS.d/next/Library/2023-07-11-08-56-40.gh-issue-106584.g…

9779a2b

…-SBtC.rst

sunmy2019 reviewed Jul 11, 2023

View reviewed changes

Lib/unittest/case.py Outdated Show resolved Hide resolved

sunmy2019 added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 11, 2023

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 11, 2023

pythonGH-106584: Apply comment suggestion

43248a9

Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>

sobolevn reviewed Jul 11, 2023

View reviewed changes

Misc/NEWS.d/next/Library/2023-07-11-08-56-40.gh-issue-106584.g-SBtC.rst Outdated Show resolved Hide resolved

EliseevEgor and others added 3 commits July 11, 2023 14:04

pythonGH-106584: Remove the version info in the NEWS

901cf01

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

pythonGH-106584: Remove empty line in the NEWS

1ab5cfa

pythonGH-106584: Check buffers before seek

6313c98

sunmy2019 added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 11, 2023

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 11, 2023

Merge branch 'main' into egoreliseev/Issue-106584_move_start_test

046ae78

terryjreedy mentioned this pull request Oct 15, 2023

unittest.TestResult.testsRun no longer counts skipped tests. #110890

Closed

felixxm mentioned this pull request Oct 15, 2023

Refs #34900, Refs #34118 -- Updated assertion in test_skip_class_unless_db_feature() test on Python 3.12.1+. django/django#17367

Merged

mtreinish mentioned this pull request Jan 5, 2024

Handle breaking api change in Python 3.12.1 testing-cabal/testtools#371

Merged

copybara-service bot mentioned this pull request Jan 11, 2024

Fix absl.testing.xml_reporter for Python 3.12.1 when all tests are skipped. abseil/abseil-py#271

Merged

wshanks mentioned this pull request Jan 19, 2024

Add Python 3.12 builds conda-forge/qiskit-aer-feedstock#31

Merged

5 tasks

serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Jan 23, 2024

Revert "pythongh-106584: Fix exit code for unittest in Python 3.12 (p…

15d1968

…ython#106588)" This reverts commit 8fc0713.

serhiy-storchaka added a commit that referenced this pull request Feb 4, 2024

gh-113267: Revert "gh-106584: Fix exit code for unittest in Python 3.…

ecabff9

…12 (#106588)" (GH-114470) This reverts commit 8fc0713.

aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024

pythongh-113267: Revert "pythongh-106584: Fix exit code for unittest …

9d44784

…in Python 3.12 (python#106588)" (pythonGH-114470) This reverts commit 8fc0713.

fsc-eriker pushed a commit to fsc-eriker/cpython that referenced this pull request Feb 14, 2024

pythongh-113267: Revert "pythongh-106584: Fix exit code for unittest …

39378cc

…in Python 3.12 (python#106588)" (pythonGH-114470) This reverts commit 8fc0713.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-106584: Fix exit code for unittest in Python 3.12 #106588

gh-106584: Fix exit code for unittest in Python 3.12 #106588

EliseevEgor commented Jul 10, 2023 •

edited by terryjreedy

Loading

bedevere-bot commented Jul 10, 2023

cpython-cla-bot bot commented Jul 10, 2023 •

edited

Loading

bedevere-bot commented Jul 10, 2023

bedevere-bot commented Jul 10, 2023

bedevere-bot commented Jul 10, 2023

sobolevn left a comment

bedevere-bot commented Jul 11, 2023

sobolevn left a comment

bedevere-bot commented Jul 11, 2023

sunmy2019 commented Jul 13, 2023

exarkun commented Dec 18, 2023

yilei commented Jan 11, 2024

gh-106584: Fix exit code for unittest in Python 3.12 #106588

gh-106584: Fix exit code for unittest in Python 3.12 #106588

Conversation

EliseevEgor commented Jul 10, 2023 • edited by terryjreedy Loading

bedevere-bot commented Jul 10, 2023

cpython-cla-bot bot commented Jul 10, 2023 • edited Loading

bedevere-bot commented Jul 10, 2023

bedevere-bot commented Jul 10, 2023

bedevere-bot commented Jul 10, 2023

sobolevn left a comment

Choose a reason for hiding this comment

bedevere-bot commented Jul 11, 2023

sobolevn left a comment

Choose a reason for hiding this comment

bedevere-bot commented Jul 11, 2023

sunmy2019 commented Jul 13, 2023

exarkun commented Dec 18, 2023

yilei commented Jan 11, 2024

EliseevEgor commented Jul 10, 2023 •

edited by terryjreedy

Loading

cpython-cla-bot bot commented Jul 10, 2023 •

edited

Loading