Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PGO tests fail on 3.11 and 3.12 #111929

Closed
neonene opened this issue Nov 10, 2023 · 5 comments
Closed

PGO tests fail on 3.11 and 3.12 #111929

neonene opened this issue Nov 10, 2023 · 5 comments
Labels
type-bug An unexpected behavior, bug, or error

Comments

@neonene
Copy link
Contributor

neonene commented Nov 10, 2023 •
edited by bedevere-app bot
Loading

Bug report

Bug description:

Recent PGO tests on Python 3.11 and 3.12 result in failure.
For example, Linux Buildbots' logs (compile section) are like:

3.12 (AMD64 RHEL7 LTO + PGO)

Using random seed: 2130237982
0:00:00 load avg: 1.75 Run 44 tests sequentially (timeout: 20 min)
0:00:00 load avg: 1.75 [ 1/44] test_array
0:00:01 load avg: 1.75 [ 2/44] test_base64
0:00:02 load avg: 1.75 [ 3/44] test_binascii
0:00:02 load avg: 1.75 [ 4/44] test_binop
0:00:02 load avg: 1.75 [ 5/44] test_bisect
0:00:03 load avg: 1.75 [ 6/44] test_bytes
0:00:15 load avg: 1.58 [ 7/44] test_bz2
0:00:16 load avg: 1.58 [ 8/44] test_cmath
0:00:16 load avg: 1.58 [ 9/44] test_codecs
0:00:19 load avg: 1.54 [10/44] test_collections
0:00:22 load avg: 1.54 [11/44] test_complex
0:00:22 load avg: 1.54 [12/44] test_dataclasses
0:00:23 load avg: 1.49 [13/44] test_datetime
0:00:32 load avg: 1.45 [14/44] test_decimal
0:00:42 load avg: 1.38 [15/44] test_difflib
0:00:45 load avg: 1.35 [16/44] test_embed
0:01:00 load avg: 1.12 [17/44] test_float
0:01:00 load avg: 1.12 [18/44] test_fstring
0:01:03 load avg: 1.12 [19/44] test_functools
0:01:04 load avg: 1.11 [20/44] test_generators
0:01:04 load avg: 1.11 [21/44] test_hashlib
0:01:06 load avg: 1.11 [22/44] test_heapq
0:01:08 load avg: 1.11 [23/44] test_int
0:01:09 load avg: 1.10 [24/44] test_itertools
0:01:20 load avg: 1.09 [25/44] test_json
0:01:25 load avg: 1.00 [26/44] test_long
0:01:33 load avg: 1.00 [27/44] test_lzma
0:01:34 load avg: 1.00 [28/44] test_math
0:01:42 load avg: 1.00 [29/44] test_memoryview
0:01:43 load avg: 1.00 [30/44] test_operator
0:01:43 load avg: 1.00 [31/44] test_ordered_dict
0:01:45 load avg: 1.00 [32/44] test_patma
0:01:46 load avg: 1.00 [33/44] test_pickle
0:02:01 load avg: 1.00 [34/44] test_pprint
0:02:02 load avg: 1.00 [35/44] test_re
0:02:04 load avg: 1.00 [36/44] test_set
0:02:18 load avg: 1.00 [37/44] test_sqlite3
0:02:19 load avg: 1.00 [38/44] test_statistics
0:02:25 load avg: 1.00 [39/44] test_str
0:02:25 load avg: 1.00 [40/44] test_struct -- test_str failed (uncaught exception)
0:02:26 load avg: 1.00 [41/44] test_tabnanny
0:02:27 load avg: 1.00 [42/44] test_time
0:02:30 load avg: 0.92 [43/44] test_xml_etree
0:02:31 load avg: 0.92 [44/44] test_xml_etree_c

Total duration: 2 min 33 sec
Total tests: run=8,699 skipped=193
Total test files: run=44/44 failed=1
Result: FAILURE

3.11 (AMD64 RHEL7 LTO + PGO)

Using random seed: 2899545713
0:00:00 load avg: 2.37 Run 44 tests sequentially (timeout: 20 min)
0:00:00 load avg: 2.37 [ 1/44] test_array
0:00:02 load avg: 2.37 [ 2/44] test_base64
0:00:02 load avg: 2.37 [ 3/44] test_binascii
0:00:03 load avg: 2.37 [ 4/44] test_binop
0:00:03 load avg: 2.37 [ 5/44] test_bisect
0:00:03 load avg: 2.37 [ 6/44] test_bytes
0:00:12 load avg: 2.16 [ 7/44] test_bz2
0:00:13 load avg: 2.16 [ 8/44] test_cmath
0:00:14 load avg: 2.16 [ 9/44] test_codecs
0:00:14 load avg: 2.16 [10/44] test_collections -- test_codecs failed (uncaught exception)
0:00:14 load avg: 2.16 [11/44] test_complex -- test_collections failed (uncaught exception)
0:00:14 load avg: 2.16 [12/44] test_dataclasses -- test_complex failed (uncaught exception)
0:00:14 load avg: 2.16 [13/44] test_datetime -- test_dataclasses failed (uncaught exception)
0:00:14 load avg: 2.16 [14/44] test_decimal -- test_datetime failed (uncaught exception)
0:00:14 load avg: 2.16 [15/44] test_difflib -- test_decimal failed (uncaught exception)
0:00:14 load avg: 2.16 [16/44] test_embed -- test_difflib failed (uncaught exception)
0:00:14 load avg: 2.16 [17/44] test_float -- test_embed failed (uncaught exception)
0:00:14 load avg: 2.16 [18/44] test_fstring -- test_float failed (uncaught exception)
0:00:14 load avg: 2.16 [19/44] test_functools -- test_fstring failed (uncaught exception)
0:00:14 load avg: 2.16 [20/44] test_generators -- test_functools failed (uncaught exception)
0:00:14 load avg: 2.16 [21/44] test_hashlib -- test_generators failed (uncaught exception)
0:00:14 load avg: 2.16 [22/44] test_heapq -- test_hashlib failed (uncaught exception)
0:00:14 load avg: 2.16 [23/44] test_int -- test_heapq failed (uncaught exception)
0:00:14 load avg: 2.16 [24/44] test_itertools -- test_int failed (uncaught exception)
0:00:14 load avg: 2.16 [25/44] test_json -- test_itertools failed (uncaught exception)
0:00:14 load avg: 2.16 [26/44] test_long -- test_json failed (uncaught exception)
0:00:14 load avg: 2.16 [27/44] test_lzma -- test_long failed (uncaught exception)
0:00:14 load avg: 2.16 [28/44] test_math -- test_lzma failed (uncaught exception)
0:00:14 load avg: 2.16 [29/44] test_memoryview -- test_math failed (uncaught exception)
0:00:14 load avg: 2.16 [30/44] test_operator -- test_memoryview failed (uncaught exception)
0:00:14 load avg: 2.16 [31/44] test_ordered_dict -- test_operator failed (uncaught exception)
0:00:14 load avg: 2.16 [32/44] test_patma -- test_ordered_dict failed (uncaught exception)
0:00:14 load avg: 2.16 [33/44] test_pickle -- test_patma failed (uncaught exception)
0:00:14 load avg: 2.16 [34/44] test_pprint -- test_pickle failed (uncaught exception)
0:00:14 load avg: 2.16 [35/44] test_re -- test_pprint failed (uncaught exception)
0:00:14 load avg: 2.16 [36/44] test_set -- test_re failed (uncaught exception)
0:00:14 load avg: 2.16 [37/44] test_sqlite3 -- test_set failed (uncaught exception)
0:00:14 load avg: 2.16 [38/44] test_statistics -- test_sqlite3 failed (uncaught exception)
0:00:14 load avg: 2.16 [39/44] test_str -- test_statistics failed (uncaught exception)
0:00:14 load avg: 2.16 [40/44] test_struct -- test_str failed (uncaught exception)
0:00:14 load avg: 2.16 [41/44] test_tabnanny -- test_struct failed (uncaught exception)
0:00:14 load avg: 2.16 [42/44] test_time -- test_tabnanny failed (uncaught exception)
0:00:14 load avg: 2.16 [43/44] test_xml_etree -- test_time failed (uncaught exception)
0:00:14 load avg: 2.16 [44/44] test_xml_etree_c -- test_xml_etree failed (uncaught exception)
test_xml_etree_c failed (uncaught exception)

Total duration: 14.2 sec
Total tests: run=1,388 skipped=9
Total test files: run=44/44 failed=36
Result: FAILURE

Seems related to issue gh-110276, gh-110756.
cc: @vstinner @AlexWaygood

CPython versions tested on:

3.12

Operating systems tested on:

Windows

Linked PRs
@neonene neonene added the type-bug An unexpected behavior, bug, or error label Nov 10, 2023
@bedevere-app bedevere-app bot mentioned this issue Nov 10, 2023
Merged
vstinner added a commit to vstinner/cpython that referenced this issue Nov 10, 2023
@vstinner
pythongh-111929: Fix regrtest --pgo: test_str => test_unicode
7b9cd0e 
test_unicode was renamed to test_str in Python 3.13, but Python 3.12
still uses test_unicode name.
vstinner added a commit that referenced this issue Nov 10, 2023
@vstinner
gh-111929: Fix regrtest --pgo: test_str => test_unicode (#111938)
5f42a2b 
test_unicode was renamed to test_str in Python 3.13, but Python 3.12
still uses test_unicode name.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Nov 10, 2023
@vstinner @miss-islington
pythongh-111929: Fix regrtest --pgo: test_str => test_unicode (python…
3baa4b8 
…GH-111938)

test_unicode was renamed to test_str in Python 3.13, but Python 3.12
still uses test_unicode name.
(cherry picked from commit 5f42a2b)

Co-authored-by: Victor Stinner <vstinner@python.org>
@bedevere-app bedevere-app bot mentioned this issue Nov 10, 2023
Merged
@vstinner
Copy link
Member

I'm working on fixes.

@AlexWaygood
Copy link
Member

AlexWaygood commented Nov 10, 2023 •
edited
Loading

How do we ensure this isn't repeated the next time we try to sync libregrtest in the backport branches with the code on the main branch?

vstinner added a commit that referenced this issue Nov 10, 2023
@miss-islington @vstinner
[3.11] gh-111929: Fix regrtest --pgo: test_str => test_unicode (GH-11…
ecc8df2 
…1938) (#111940)

gh-111929: Fix regrtest --pgo: test_str => test_unicode (GH-111938)

test_unicode was renamed to test_str in Python 3.13, but Python 3.12
still uses test_unicode name.
(cherry picked from commit 5f42a2b)

Co-authored-by: Victor Stinner <vstinner@python.org>
@vstinner
Copy link
Member

I'm considering to backport the Makefile change which treats a failure... as a failure: don't ignore the failure.

@bedevere-app bedevere-app bot mentioned this issue Nov 10, 2023
Merged
@vstinner vstinner mentioned this issue Nov 10, 2023
Closed
@vstinner
Copy link
Member

I'm considering to backport the Makefile change which treats a failure... as a failure: don't ignore the failure.

I created #111950 for that.

vstinner added a commit that referenced this issue Nov 12, 2023
@vstinner
[3.11] gh-111929: Fix regrtest clear_caches() (#111949)
aa3232d 
gh-111929: Fix regrtest clear_caches()

Python 3.11 doesn't have the fractions._hash_algorithm.cache_clear()
function.
@neonene neonene closed this as completed Nov 15, 2023
@vstinner
Copy link
Member

Thanks @neonene for your bug report!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@vstinner @neonene @AlexWaygood