Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Freethreading build fails -m test --pgo: test_ordered_dict and test_struct failed #118846

Closed
hroncok opened this issue May 9, 2024 · 4 comments
Closed

Freethreading build fails -m test --pgo: test_ordered_dict and test_struct failed #118846

hroncok opened this issue May 9, 2024 · 4 comments
Assignees
@colesbury
Labels
tests Tests in the Lib/test dir topic-free-threading type-bug An unexpected behavior, bug, or error

Comments

@hroncok
Copy link
Contributor

hroncok commented May 9, 2024 •
edited by bedevere-app bot
Loading

Bug report

Bug description:

When we build Python 3.13.0b1 in Fedora with --disable-gil --enable-optimizations, the PGO task fails. I was able to reproduce this from the 3.13 branch by:

$ ./configure --config-cache --without-ensurepip --disable-gil --enable-optimizations
$ make
...
# Next, run the profile task to generate the profile information.
./python -m test --pgo --timeout=
Using random seed: 754312824
0:00:00 load avg: 8.58 Run 44 tests sequentially
0:00:00 load avg: 8.58 [ 1/44] test_array
0:00:01 load avg: 8.58 [ 2/44] test_base64
0:00:01 load avg: 8.58 [ 3/44] test_binascii
0:00:01 load avg: 8.58 [ 4/44] test_binop
0:00:01 load avg: 8.58 [ 5/44] test_bisect
0:00:01 load avg: 8.58 [ 6/44] test_bytes
0:00:07 load avg: 8.13 [ 7/44] test_bz2
0:00:08 load avg: 7.88 [ 8/44] test_cmath
0:00:08 load avg: 7.88 [ 9/44] test_codecs
0:00:10 load avg: 7.88 [10/44] test_collections
0:00:12 load avg: 7.88 [11/44] test_complex
0:00:12 load avg: 7.88 [12/44] test_dataclasses
0:00:13 load avg: 7.57 [13/44] test_datetime
0:00:19 load avg: 7.04 [14/44] test_decimal
0:00:24 load avg: 6.56 [15/44] test_difflib
0:00:25 load avg: 6.56 [16/44] test_embed
0:00:31 load avg: 6.11 [17/44] test_float
0:00:31 load avg: 6.11 [18/44] test_fstring
0:00:33 load avg: 6.10 [19/44] test_functools
0:00:35 load avg: 6.10 [20/44] test_generators
0:00:37 load avg: 6.10 [21/44] test_hashlib
0:00:38 load avg: 6.10 [22/44] test_heapq
0:00:39 load avg: 5.77 [23/44] test_int
0:00:40 load avg: 5.77 [24/44] test_itertools
0:00:48 load avg: 5.04 [25/44] test_json
0:00:54 load avg: 4.80 [26/44] test_long
0:00:59 load avg: 4.65 [27/44] test_lzma
0:01:00 load avg: 4.65 [28/44] test_math
0:01:06 load avg: 4.84 [29/44] test_memoryview
0:01:08 load avg: 4.77 [30/44] test_operator
0:01:08 load avg: 4.77 [31/44] test_ordered_dict
test test_ordered_dict failed
0:01:12 load avg: 4.77 [32/44] test_patma -- test_ordered_dict failed (4 failures)
0:01:12 load avg: 4.77 [33/44] test_pickle
0:01:21 load avg: 4.50 [34/44] test_pprint
0:01:22 load avg: 4.50 [35/44] test_re
0:01:23 load avg: 4.38 [36/44] test_set
0:01:33 load avg: 3.94 [37/44] test_sqlite3
0:01:34 load avg: 3.94 [38/44] test_statistics -- test_sqlite3 skipped
0:01:48 load avg: 3.58 [39/44] test_str
0:01:57 load avg: 3.53 [40/44] test_struct
test test_struct failed
0:02:01 load avg: 3.57 [41/44] test_tabnanny -- test_struct failed (1 failure)
0:02:05 load avg: 3.36 [42/44] test_time
0:02:09 load avg: 3.33 [43/44] test_xml_etree
0:02:20 load avg: 3.21 [44/44] test_xml_etree_c

Total duration: 2 min 29 sec
Total tests: run=8,756 failures=5 skipped=187
Total test files: run=44/44 failed=2 skipped=1
Result: FAILURE

Interestingly, the tests pass when run individually:

$ ./python -m test --pgo test_ordered_dict
Using random seed: 1047184953
0:00:00 load avg: 2.98 Run 1 test sequentially
0:00:00 load avg: 2.98 [1/1] test_ordered_dict

Total duration: 769 ms
Total tests: run=283
Total test files: run=1/1
Result: SUCCESS

$ ./python -m test --pgo test_struct
Using random seed: 2588757208
0:00:00 load avg: 2.98 Run 1 test sequentially
0:00:00 load avg: 2.98 [1/1] test_struct

Total duration: 567 ms
Total tests: run=39
Total test files: run=1/1
Result: SUCCESS

But not when all of them run:

$ ./python -m test --pgo 
Using random seed: 3047572841
0:00:00 load avg: 2.98 Run 44 tests sequentially
0:00:00 load avg: 2.98 [ 1/44] test_array
0:00:00 load avg: 2.98 [ 2/44] test_base64
0:00:01 load avg: 2.98 [ 3/44] test_binascii
0:00:01 load avg: 2.98 [ 4/44] test_binop
0:00:01 load avg: 2.98 [ 5/44] test_bisect
0:00:01 load avg: 2.98 [ 6/44] test_bytes
0:00:05 load avg: 2.98 [ 7/44] test_bz2
0:00:06 load avg: 2.98 [ 8/44] test_cmath
0:00:06 load avg: 2.98 [ 9/44] test_codecs
0:00:08 load avg: 2.82 [10/44] test_collections
0:00:10 load avg: 2.82 [11/44] test_complex
0:00:10 load avg: 2.82 [12/44] test_dataclasses
0:00:11 load avg: 2.82 [13/44] test_datetime
0:00:16 load avg: 3.16 [14/44] test_decimal
0:00:21 load avg: 3.14 [15/44] test_difflib
0:00:22 load avg: 3.14 [16/44] test_embed
0:00:33 load avg: 3.19 [17/44] test_float
0:00:34 load avg: 3.19 [18/44] test_fstring
0:00:36 load avg: 3.19 [19/44] test_functools
0:00:38 load avg: 3.18 [20/44] test_generators
0:00:39 load avg: 3.18 [21/44] test_hashlib
0:00:40 load avg: 3.18 [22/44] test_heapq
0:00:42 load avg: 3.18 [23/44] test_int
0:00:44 load avg: 3.24 [24/44] test_itertools
0:00:54 load avg: 3.28 [25/44] test_json
0:00:59 load avg: 3.34 [26/44] test_long
0:01:03 load avg: 3.39 [27/44] test_lzma
0:01:03 load avg: 3.39 [28/44] test_math
0:01:07 load avg: 3.39 [29/44] test_memoryview
0:01:08 load avg: 3.36 [30/44] test_operator
0:01:09 load avg: 3.36 [31/44] test_ordered_dict
test test_ordered_dict failed
0:01:12 load avg: 3.25 [32/44] test_patma -- test_ordered_dict failed (4 failures)
0:01:13 load avg: 3.25 [33/44] test_pickle
0:01:21 load avg: 3.23 [34/44] test_pprint
0:01:21 load avg: 3.23 [35/44] test_re
0:01:23 load avg: 3.45 [36/44] test_set
0:01:33 load avg: 3.15 [37/44] test_sqlite3
0:01:35 load avg: 3.15 [38/44] test_statistics -- test_sqlite3 skipped
0:02:00 load avg: 4.33 [39/44] test_str
0:02:07 load avg: 4.30 [40/44] test_struct
test test_struct failed
0:02:10 load avg: 4.28 [41/44] test_tabnanny -- test_struct failed (1 failure)
0:02:12 load avg: 4.10 [42/44] test_time
0:02:16 load avg: 4.10 [43/44] test_xml_etree
0:02:23 load avg: 3.93 [44/44] test_xml_etree_c

Total duration: 2 min 33 sec
Total tests: run=8,756 failures=5 skipped=187
Total test files: run=44/44 failed=2 skipped=1
Result: FAILURE

I don't know what the failure is, as using -v is not possible:

$ ./python -m test --pgo -v
usage: python -m test [options] [test_name1 [test_name2 ...]]
python path/to/Lib/test/regrtest.py [options] [test_name1 [test_name2 ...]]
regrtest.py: error: --pgo/-v don't go together!
Pass -h or --help for complete help.

I can however run the pgo tests manually like this:

$ ./python -m test -W test_array test_base64 test_binascii test_binop test_bisect test_bytes test_bz2 test_cmath test_codecs test_collections test_complex test_dataclasses test_datetime test_decimal test_difflib test_embed test_float test_fstring test_functools test_generators test_hashlib test_heapq test_int test_itertools test_json test_long test_lzma test_math test_memoryview test_operator test_ordered_dict test_patma test_pickle test_pprint test_re test_set test_sqlite3 test_statistics test_str test_struct test_tabnanny test_time test_xml_etree test_xml_etree_c 
Using random seed: 2816723097
0:00:00 load avg: 2.57 Run 44 tests sequentially
0:00:00 load avg: 2.57 [ 1/44] test_array
0:00:02 load avg: 2.57 [ 2/44] test_base64
0:00:02 load avg: 2.57 [ 3/44] test_binascii
0:00:02 load avg: 2.57 [ 4/44] test_binop
0:00:02 load avg: 2.57 [ 5/44] test_bisect
0:00:02 load avg: 2.57 [ 6/44] test_bytes
0:00:07 load avg: 2.44 [ 7/44] test_bz2
0:00:08 load avg: 2.41 [ 8/44] test_cmath
0:00:08 load avg: 2.41 [ 9/44] test_codecs
0:00:10 load avg: 2.41 [10/44] test_collections
0:00:12 load avg: 2.41 [11/44] test_complex
0:00:12 load avg: 2.41 [12/44] test_dataclasses
0:00:13 load avg: 2.93 [13/44] test_datetime
0:00:19 load avg: 3.18 [14/44] test_decimal
0:00:24 load avg: 3.25 [15/44] test_difflib
0:00:26 load avg: 3.25 [16/44] test_embed
0:00:33 load avg: 3.37 [17/44] test_float
0:00:33 load avg: 3.37 [18/44] test_fstring
0:00:34 load avg: 3.37 [19/44] test_functools
0:00:36 load avg: 3.37 [20/44] test_generators
0:00:36 load avg: 3.37 [21/44] test_hashlib
0:00:37 load avg: 3.37 [22/44] test_heapq
0:00:38 load avg: 3.26 [23/44] test_int
0:00:39 load avg: 3.26 [24/44] test_itertools
0:00:49 load avg: 3.54 [25/44] test_json
0:00:58 load avg: 3.68 [26/44] test_long
0:01:05 load avg: 3.78 [27/44] test_lzma
0:01:06 load avg: 3.78 [28/44] test_math
0:01:14 load avg: 3.97 [29/44] test_memoryview
0:01:16 load avg: 3.97 [30/44] test_operator
0:01:17 load avg: 3.97 [31/44] test_ordered_dict
test_abc (test.test_ordered_dict.CPythonBuiltinDictTests.test_abc) ... ok
test_clear (test.test_ordered_dict.CPythonBuiltinDictTests.test_clear) ... ok
test_delitem (test.test_ordered_dict.CPythonBuiltinDictTests.test_delitem) ... ok
test_delitem_hash_collision (test.test_ordered_dict.CPythonBuiltinDictTests.test_delitem_hash_collision) ... ok
test_detect_deletion_during_iteration (test.test_ordered_dict.CPythonBuiltinDictTests.test_detect_deletion_during_iteration) ... ok
test_highly_nested (test.test_ordered_dict.CPythonBuiltinDictTests.test_highly_nested) ... ok
test_highly_nested_subclass (test.test_ordered_dict.CPythonBuiltinDictTests.test_highly_nested_subclass) ... ok
test_init (test.test_ordered_dict.CPythonBuiltinDictTests.test_init) ... ok
test_override_update (test.test_ordered_dict.CPythonBuiltinDictTests.test_override_update) ... ok
test_popitem (test.test_ordered_dict.CPythonBuiltinDictTests.test_popitem) ... ok
test_reinsert (test.test_ordered_dict.CPythonBuiltinDictTests.test_reinsert) ... ok
test_setitem (test.test_ordered_dict.CPythonBuiltinDictTests.test_setitem) ... ok
test_update (test.test_ordered_dict.CPythonBuiltinDictTests.test_update) ... ok
test_bool (test.test_ordered_dict.CPythonGeneralMappingTests.test_bool) ... ok
test_constructor (test.test_ordered_dict.CPythonGeneralMappingTests.test_constructor) ... ok
test_get (test.test_ordered_dict.CPythonGeneralMappingTests.test_get) ... ok
test_getitem (test.test_ordered_dict.CPythonGeneralMappingTests.test_getitem) ... ok
test_items (test.test_ordered_dict.CPythonGeneralMappingTests.test_items) ... ok
test_keys (test.test_ordered_dict.CPythonGeneralMappingTests.test_keys) ... ok
test_len (test.test_ordered_dict.CPythonGeneralMappingTests.test_len) ... ok
test_pop (test.test_ordered_dict.CPythonGeneralMappingTests.test_pop) ... ok
test_popitem (test.test_ordered_dict.CPythonGeneralMappingTests.test_popitem) ... ok
test_read (test.test_ordered_dict.CPythonGeneralMappingTests.test_read) ... ok
test_setdefault (test.test_ordered_dict.CPythonGeneralMappingTests.test_setdefault) ... ok
test_update (test.test_ordered_dict.CPythonGeneralMappingTests.test_update) ... ok
test_values (test.test_ordered_dict.CPythonGeneralMappingTests.test_values) ... ok
test_write (test.test_ordered_dict.CPythonGeneralMappingTests.test_write) ... ok
test_468 (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_468) ... ok
test_abc (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_abc) ... ok
test_clear (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_clear) ... ok
test_copying (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_copying) ... ok
test_delitem (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_delitem) ... ok
test_delitem_hash_collision (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_delitem_hash_collision) ... ok
test_detect_deletion_during_iteration (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_detect_deletion_during_iteration) ... ok
test_dict_clear (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_dict_clear) ... ok
test_dict_delitem (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_dict_delitem) ... ok
test_dict_pop (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_dict_pop) ... ok
test_dict_popitem (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_dict_popitem) ... ok
test_dict_setdefault (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_dict_setdefault) ... ok
test_dict_setitem (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_dict_setitem) ... ok
test_dict_update (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_dict_update) ... ok
test_equality (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_equality) ... ok
test_free_after_iterating (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_free_after_iterating) ... ok
test_fromkeys (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_fromkeys) ... ok
test_highly_nested (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_highly_nested) ... ok
test_highly_nested_subclass (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_highly_nested_subclass) ... ok
test_init (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_init) ... ok
test_init_calls (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_init_calls) ... ok
test_issue24347 (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_issue24347) ... ok
test_issue24348 (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_issue24348) ... ok
test_issue24667 (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_issue24667)
dict resizes after a certain number of insertion operations, ... ok
test_iterators (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_iterators) ... ok
test_iterators_empty (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_iterators_empty) ... ok
test_iterators_pickling (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_iterators_pickling) ... ok
test_key_change_during_iteration (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_key_change_during_iteration) ... ok
test_merge_operator (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_merge_operator) ... ok
test_move_to_end (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_move_to_end) ... ok
test_move_to_end_issue25406 (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_move_to_end_issue25406) ... ok
test_ordered_dict_items_result_gc (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_ordered_dict_items_result_gc) ... ok
test_overridden_init (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_overridden_init) ... ok
test_override_update (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_override_update) ... ok
test_pickle_recursive (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_pickle_recursive) ... ok
test_pop (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_pop) ... ok
test_popitem (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_popitem) ... ok
test_popitem_last (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_popitem_last) ... ok
test_reduce_not_too_fat (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_reduce_not_too_fat) ... ok
test_reference_loop (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_reference_loop) ... FAIL
test_reinsert (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_reinsert) ... ok
test_repr (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_repr) ... ok
test_repr_recursive (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_repr_recursive) ... ok
test_repr_recursive_values (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_repr_recursive_values) ... ok
test_setdefault (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_setdefault) ... ok
test_setitem (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_setitem) ... ok
test_sizeof (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_sizeof) ... ok
test_sizeof_exact (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_sizeof_exact) ... ok
test_sorted_iterators (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_sorted_iterators) ... ok
test_update (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_update) ... ok
test_views (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_views) ... ok
test_weakref_list_is_not_traversed (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_weakref_list_is_not_traversed) ... ok
test_yaml_linkage (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_yaml_linkage) ... ok
test_468 (test.test_ordered_dict.CPythonOrderedDictTests.test_468) ... ok
test_abc (test.test_ordered_dict.CPythonOrderedDictTests.test_abc) ... ok
test_clear (test.test_ordered_dict.CPythonOrderedDictTests.test_clear) ... ok
test_copying (test.test_ordered_dict.CPythonOrderedDictTests.test_copying) ... ok
test_delitem (test.test_ordered_dict.CPythonOrderedDictTests.test_delitem) ... ok
test_delitem_hash_collision (test.test_ordered_dict.CPythonOrderedDictTests.test_delitem_hash_collision) ... ok
test_detect_deletion_during_iteration (test.test_ordered_dict.CPythonOrderedDictTests.test_detect_deletion_during_iteration) ... ok
test_dict_clear (test.test_ordered_dict.CPythonOrderedDictTests.test_dict_clear) ... ok
test_dict_delitem (test.test_ordered_dict.CPythonOrderedDictTests.test_dict_delitem) ... ok
test_dict_pop (test.test_ordered_dict.CPythonOrderedDictTests.test_dict_pop) ... ok
test_dict_popitem (test.test_ordered_dict.CPythonOrderedDictTests.test_dict_popitem) ... ok
test_dict_setdefault (test.test_ordered_dict.CPythonOrderedDictTests.test_dict_setdefault) ... ok
test_dict_setitem (test.test_ordered_dict.CPythonOrderedDictTests.test_dict_setitem) ... ok
test_dict_update (test.test_ordered_dict.CPythonOrderedDictTests.test_dict_update) ... ok
test_equality (test.test_ordered_dict.CPythonOrderedDictTests.test_equality) ... ok
test_free_after_iterating (test.test_ordered_dict.CPythonOrderedDictTests.test_free_after_iterating) ... ok
test_fromkeys (test.test_ordered_dict.CPythonOrderedDictTests.test_fromkeys) ... ok
test_highly_nested (test.test_ordered_dict.CPythonOrderedDictTests.test_highly_nested) ... ok
test_highly_nested_subclass (test.test_ordered_dict.CPythonOrderedDictTests.test_highly_nested_subclass) ... ok
test_init (test.test_ordered_dict.CPythonOrderedDictTests.test_init) ... ok
test_init_calls (test.test_ordered_dict.CPythonOrderedDictTests.test_init_calls) ... ok
test_issue24347 (test.test_ordered_dict.CPythonOrderedDictTests.test_issue24347) ... ok
test_issue24348 (test.test_ordered_dict.CPythonOrderedDictTests.test_issue24348) ... ok
test_issue24667 (test.test_ordered_dict.CPythonOrderedDictTests.test_issue24667)
dict resizes after a certain number of insertion operations, ... ok
test_iterators (test.test_ordered_dict.CPythonOrderedDictTests.test_iterators) ... ok
test_iterators_empty (test.test_ordered_dict.CPythonOrderedDictTests.test_iterators_empty) ... ok
test_iterators_pickling (test.test_ordered_dict.CPythonOrderedDictTests.test_iterators_pickling) ... ok
test_key_change_during_iteration (test.test_ordered_dict.CPythonOrderedDictTests.test_key_change_during_iteration) ... ok
test_merge_operator (test.test_ordered_dict.CPythonOrderedDictTests.test_merge_operator) ... ok
test_move_to_end (test.test_ordered_dict.CPythonOrderedDictTests.test_move_to_end) ... ok
test_move_to_end_issue25406 (test.test_ordered_dict.CPythonOrderedDictTests.test_move_to_end_issue25406) ... ok
test_ordered_dict_items_result_gc (test.test_ordered_dict.CPythonOrderedDictTests.test_ordered_dict_items_result_gc) ... ok
test_overridden_init (test.test_ordered_dict.CPythonOrderedDictTests.test_overridden_init) ... ok
test_override_update (test.test_ordered_dict.CPythonOrderedDictTests.test_override_update) ... ok
test_pickle_recursive (test.test_ordered_dict.CPythonOrderedDictTests.test_pickle_recursive) ... ok
test_pop (test.test_ordered_dict.CPythonOrderedDictTests.test_pop) ... ok
test_popitem (test.test_ordered_dict.CPythonOrderedDictTests.test_popitem) ... ok
test_popitem_last (test.test_ordered_dict.CPythonOrderedDictTests.test_popitem_last) ... ok
test_reduce_not_too_fat (test.test_ordered_dict.CPythonOrderedDictTests.test_reduce_not_too_fat) ... ok
test_reference_loop (test.test_ordered_dict.CPythonOrderedDictTests.test_reference_loop) ... FAIL
test_reinsert (test.test_ordered_dict.CPythonOrderedDictTests.test_reinsert) ... ok
test_repr (test.test_ordered_dict.CPythonOrderedDictTests.test_repr) ... ok
test_repr_recursive (test.test_ordered_dict.CPythonOrderedDictTests.test_repr_recursive) ... ok
test_repr_recursive_values (test.test_ordered_dict.CPythonOrderedDictTests.test_repr_recursive_values) ... ok
test_setdefault (test.test_ordered_dict.CPythonOrderedDictTests.test_setdefault) ... ok
test_setitem (test.test_ordered_dict.CPythonOrderedDictTests.test_setitem) ... ok
test_sizeof (test.test_ordered_dict.CPythonOrderedDictTests.test_sizeof) ... ok
test_sizeof_exact (test.test_ordered_dict.CPythonOrderedDictTests.test_sizeof_exact) ... ok
test_sorted_iterators (test.test_ordered_dict.CPythonOrderedDictTests.test_sorted_iterators) ... ok
test_update (test.test_ordered_dict.CPythonOrderedDictTests.test_update) ... ok
test_views (test.test_ordered_dict.CPythonOrderedDictTests.test_views) ... ok
test_weakref_list_is_not_traversed (test.test_ordered_dict.CPythonOrderedDictTests.test_weakref_list_is_not_traversed) ... ok
test_yaml_linkage (test.test_ordered_dict.CPythonOrderedDictTests.test_yaml_linkage) ... ok
test_copying (test.test_ordered_dict.CPythonOrderedDictWithSlotsCopyingTests.test_copying) ... ok
test_bool (test.test_ordered_dict.CPythonSubclassMappingTests.test_bool) ... ok
test_constructor (test.test_ordered_dict.CPythonSubclassMappingTests.test_constructor) ... ok
test_get (test.test_ordered_dict.CPythonSubclassMappingTests.test_get) ... ok
test_getitem (test.test_ordered_dict.CPythonSubclassMappingTests.test_getitem) ... ok
test_items (test.test_ordered_dict.CPythonSubclassMappingTests.test_items) ... ok
test_keys (test.test_ordered_dict.CPythonSubclassMappingTests.test_keys) ... ok
test_len (test.test_ordered_dict.CPythonSubclassMappingTests.test_len) ... ok
test_pop (test.test_ordered_dict.CPythonSubclassMappingTests.test_pop) ... ok
test_popitem (test.test_ordered_dict.CPythonSubclassMappingTests.test_popitem) ... ok
test_read (test.test_ordered_dict.CPythonSubclassMappingTests.test_read) ... ok
test_setdefault (test.test_ordered_dict.CPythonSubclassMappingTests.test_setdefault) ... ok
test_update (test.test_ordered_dict.CPythonSubclassMappingTests.test_update) ... ok
test_values (test.test_ordered_dict.CPythonSubclassMappingTests.test_values) ... ok
test_write (test.test_ordered_dict.CPythonSubclassMappingTests.test_write) ... ok
test_add_after_full (test.test_ordered_dict.CSimpleLRUCacheTests.test_add_after_full) ... ok
test_change_order_on_get (test.test_ordered_dict.CSimpleLRUCacheTests.test_change_order_on_get) ... ok
test_pop (test.test_ordered_dict.CSimpleLRUCacheTests.test_pop) ... ok
test_popitem (test.test_ordered_dict.CSimpleLRUCacheTests.test_popitem) ... ok
test_bool (test.test_ordered_dict.PurePythonGeneralMappingTests.test_bool) ... ok
test_constructor (test.test_ordered_dict.PurePythonGeneralMappingTests.test_constructor) ... ok
test_get (test.test_ordered_dict.PurePythonGeneralMappingTests.test_get) ... ok
test_getitem (test.test_ordered_dict.PurePythonGeneralMappingTests.test_getitem) ... ok
test_items (test.test_ordered_dict.PurePythonGeneralMappingTests.test_items) ... ok
test_keys (test.test_ordered_dict.PurePythonGeneralMappingTests.test_keys) ... ok
test_len (test.test_ordered_dict.PurePythonGeneralMappingTests.test_len) ... ok
test_pop (test.test_ordered_dict.PurePythonGeneralMappingTests.test_pop) ... ok
test_popitem (test.test_ordered_dict.PurePythonGeneralMappingTests.test_popitem) ... ok
test_read (test.test_ordered_dict.PurePythonGeneralMappingTests.test_read) ... ok
test_setdefault (test.test_ordered_dict.PurePythonGeneralMappingTests.test_setdefault) ... ok
test_update (test.test_ordered_dict.PurePythonGeneralMappingTests.test_update) ... ok
test_values (test.test_ordered_dict.PurePythonGeneralMappingTests.test_values) ... ok
test_write (test.test_ordered_dict.PurePythonGeneralMappingTests.test_write) ... ok
test_468 (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_468) ... ok
test_abc (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_abc) ... ok
test_clear (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_clear) ... ok
test_copying (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_copying) ... ok
test_delitem (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_delitem) ... ok
test_delitem_hash_collision (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_delitem_hash_collision) ... ok
test_detect_deletion_during_iteration (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_detect_deletion_during_iteration) ... ok
test_dict_clear (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_dict_clear) ... ok
test_dict_delitem (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_dict_delitem) ... ok
test_dict_pop (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_dict_pop) ... ok
test_dict_popitem (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_dict_popitem) ... ok
test_dict_setdefault (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_dict_setdefault) ... ok
test_dict_setitem (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_dict_setitem) ... ok
test_dict_update (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_dict_update) ... ok
test_equality (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_equality) ... ok
test_free_after_iterating (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_free_after_iterating) ... ok
test_fromkeys (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_fromkeys) ... ok
test_highly_nested (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_highly_nested) ... ok
test_highly_nested_subclass (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_highly_nested_subclass) ... ok
test_init (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_init) ... ok
test_init_calls (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_init_calls) ... ok
test_issue24347 (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_issue24347) ... ok
test_issue24348 (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_issue24348) ... ok
test_issue24667 (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_issue24667)
dict resizes after a certain number of insertion operations, ... ok
test_iterators (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_iterators) ... ok
test_iterators_empty (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_iterators_empty) ... ok
test_merge_operator (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_merge_operator) ... ok
test_move_to_end (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_move_to_end) ... ok
test_move_to_end_issue25406 (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_move_to_end_issue25406) ... ok
test_ordered_dict_items_result_gc (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_ordered_dict_items_result_gc) ... ok
test_overridden_init (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_overridden_init) ... ok
test_override_update (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_override_update) ... ok
test_pickle_recursive (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_pickle_recursive) ... ok
test_pop (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_pop) ... ok
test_popitem (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_popitem) ... ok
test_popitem_last (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_popitem_last) ... ok
test_reduce_not_too_fat (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_reduce_not_too_fat) ... ok
test_reference_loop (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_reference_loop) ... FAIL
test_reinsert (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_reinsert) ... ok
test_repr (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_repr) ... ok
test_repr_recursive (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_repr_recursive) ... ok
test_repr_recursive_values (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_repr_recursive_values) ... ok
test_setdefault (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_setdefault) ... ok
test_setitem (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_setitem) ... ok
test_sizeof (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_sizeof) ... ok
test_sorted_iterators (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_sorted_iterators) ... ok
test_update (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_update) ... ok
test_views (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_views) ... ok
test_yaml_linkage (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_yaml_linkage) ... ok
test_468 (test.test_ordered_dict.PurePythonOrderedDictTests.test_468) ... ok
test_abc (test.test_ordered_dict.PurePythonOrderedDictTests.test_abc) ... ok
test_clear (test.test_ordered_dict.PurePythonOrderedDictTests.test_clear) ... ok
test_copying (test.test_ordered_dict.PurePythonOrderedDictTests.test_copying) ... ok
test_delitem (test.test_ordered_dict.PurePythonOrderedDictTests.test_delitem) ... ok
test_delitem_hash_collision (test.test_ordered_dict.PurePythonOrderedDictTests.test_delitem_hash_collision) ... ok
test_detect_deletion_during_iteration (test.test_ordered_dict.PurePythonOrderedDictTests.test_detect_deletion_during_iteration) ... ok
test_dict_clear (test.test_ordered_dict.PurePythonOrderedDictTests.test_dict_clear) ... ok
test_dict_delitem (test.test_ordered_dict.PurePythonOrderedDictTests.test_dict_delitem) ... ok
test_dict_pop (test.test_ordered_dict.PurePythonOrderedDictTests.test_dict_pop) ... ok
test_dict_popitem (test.test_ordered_dict.PurePythonOrderedDictTests.test_dict_popitem) ... ok
test_dict_setdefault (test.test_ordered_dict.PurePythonOrderedDictTests.test_dict_setdefault) ... ok
test_dict_setitem (test.test_ordered_dict.PurePythonOrderedDictTests.test_dict_setitem) ... ok
test_dict_update (test.test_ordered_dict.PurePythonOrderedDictTests.test_dict_update) ... ok
test_equality (test.test_ordered_dict.PurePythonOrderedDictTests.test_equality) ... ok
test_free_after_iterating (test.test_ordered_dict.PurePythonOrderedDictTests.test_free_after_iterating) ... ok
test_fromkeys (test.test_ordered_dict.PurePythonOrderedDictTests.test_fromkeys) ... ok
test_highly_nested (test.test_ordered_dict.PurePythonOrderedDictTests.test_highly_nested) ... ok
test_highly_nested_subclass (test.test_ordered_dict.PurePythonOrderedDictTests.test_highly_nested_subclass) ... ok
test_init (test.test_ordered_dict.PurePythonOrderedDictTests.test_init) ... ok
test_init_calls (test.test_ordered_dict.PurePythonOrderedDictTests.test_init_calls) ... ok
test_issue24347 (test.test_ordered_dict.PurePythonOrderedDictTests.test_issue24347) ... ok
test_issue24348 (test.test_ordered_dict.PurePythonOrderedDictTests.test_issue24348) ... ok
test_issue24667 (test.test_ordered_dict.PurePythonOrderedDictTests.test_issue24667)
dict resizes after a certain number of insertion operations, ... ok
test_iterators (test.test_ordered_dict.PurePythonOrderedDictTests.test_iterators) ... ok
test_iterators_empty (test.test_ordered_dict.PurePythonOrderedDictTests.test_iterators_empty) ... ok
test_merge_operator (test.test_ordered_dict.PurePythonOrderedDictTests.test_merge_operator) ... ok
test_move_to_end (test.test_ordered_dict.PurePythonOrderedDictTests.test_move_to_end) ... ok
test_move_to_end_issue25406 (test.test_ordered_dict.PurePythonOrderedDictTests.test_move_to_end_issue25406) ... ok
test_ordered_dict_items_result_gc (test.test_ordered_dict.PurePythonOrderedDictTests.test_ordered_dict_items_result_gc) ... ok
test_overridden_init (test.test_ordered_dict.PurePythonOrderedDictTests.test_overridden_init) ... ok
test_override_update (test.test_ordered_dict.PurePythonOrderedDictTests.test_override_update) ... ok
test_pickle_recursive (test.test_ordered_dict.PurePythonOrderedDictTests.test_pickle_recursive) ... ok
test_pop (test.test_ordered_dict.PurePythonOrderedDictTests.test_pop) ... ok
test_popitem (test.test_ordered_dict.PurePythonOrderedDictTests.test_popitem) ... ok
test_popitem_last (test.test_ordered_dict.PurePythonOrderedDictTests.test_popitem_last) ... ok
test_reduce_not_too_fat (test.test_ordered_dict.PurePythonOrderedDictTests.test_reduce_not_too_fat) ... ok
test_reference_loop (test.test_ordered_dict.PurePythonOrderedDictTests.test_reference_loop) ... FAIL
test_reinsert (test.test_ordered_dict.PurePythonOrderedDictTests.test_reinsert) ... ok
test_repr (test.test_ordered_dict.PurePythonOrderedDictTests.test_repr) ... ok
test_repr_recursive (test.test_ordered_dict.PurePythonOrderedDictTests.test_repr_recursive) ... ok
test_repr_recursive_values (test.test_ordered_dict.PurePythonOrderedDictTests.test_repr_recursive_values) ... ok
test_setdefault (test.test_ordered_dict.PurePythonOrderedDictTests.test_setdefault) ... ok
test_setitem (test.test_ordered_dict.PurePythonOrderedDictTests.test_setitem) ... ok
test_sizeof (test.test_ordered_dict.PurePythonOrderedDictTests.test_sizeof) ... ok
test_sorted_iterators (test.test_ordered_dict.PurePythonOrderedDictTests.test_sorted_iterators) ... ok
test_update (test.test_ordered_dict.PurePythonOrderedDictTests.test_update) ... ok
test_views (test.test_ordered_dict.PurePythonOrderedDictTests.test_views) ... ok
test_yaml_linkage (test.test_ordered_dict.PurePythonOrderedDictTests.test_yaml_linkage) ... ok
test_copying (test.test_ordered_dict.PurePythonOrderedDictWithSlotsCopyingTests.test_copying) ... ok
test_bool (test.test_ordered_dict.PurePythonSubclassMappingTests.test_bool) ... ok
test_constructor (test.test_ordered_dict.PurePythonSubclassMappingTests.test_constructor) ... ok
test_get (test.test_ordered_dict.PurePythonSubclassMappingTests.test_get) ... ok
test_getitem (test.test_ordered_dict.PurePythonSubclassMappingTests.test_getitem) ... ok
test_items (test.test_ordered_dict.PurePythonSubclassMappingTests.test_items) ... ok
test_keys (test.test_ordered_dict.PurePythonSubclassMappingTests.test_keys) ... ok
test_len (test.test_ordered_dict.PurePythonSubclassMappingTests.test_len) ... ok
test_pop (test.test_ordered_dict.PurePythonSubclassMappingTests.test_pop) ... ok
test_popitem (test.test_ordered_dict.PurePythonSubclassMappingTests.test_popitem) ... ok
test_read (test.test_ordered_dict.PurePythonSubclassMappingTests.test_read) ... ok
test_setdefault (test.test_ordered_dict.PurePythonSubclassMappingTests.test_setdefault) ... ok
test_update (test.test_ordered_dict.PurePythonSubclassMappingTests.test_update) ... ok
test_values (test.test_ordered_dict.PurePythonSubclassMappingTests.test_values) ... ok
test_write (test.test_ordered_dict.PurePythonSubclassMappingTests.test_write) ... ok
test_add_after_full (test.test_ordered_dict.PySimpleLRUCacheTests.test_add_after_full) ... ok
test_change_order_on_get (test.test_ordered_dict.PySimpleLRUCacheTests.test_change_order_on_get) ... ok
test_pop (test.test_ordered_dict.PySimpleLRUCacheTests.test_pop) ... ok
test_popitem (test.test_ordered_dict.PySimpleLRUCacheTests.test_popitem) ... ok

======================================================================
FAIL: test_reference_loop (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_reference_loop)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_ordered_dict.py", line 679, in test_reference_loop
    self.assertIsNone(r())
    ~~~~~~~~~~~~~~~~~^^^^^
AssertionError: <class 'test.test_ordered_dict.OrderedDictTests.test_reference_loop.<locals>.A'> is not None

======================================================================
FAIL: test_reference_loop (test.test_ordered_dict.CPythonOrderedDictTests.test_reference_loop)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_ordered_dict.py", line 679, in test_reference_loop
    self.assertIsNone(r())
    ~~~~~~~~~~~~~~~~~^^^^^
AssertionError: <class 'test.test_ordered_dict.OrderedDictTests.test_reference_loop.<locals>.A'> is not None

======================================================================
FAIL: test_reference_loop (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_reference_loop)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_ordered_dict.py", line 679, in test_reference_loop
    self.assertIsNone(r())
    ~~~~~~~~~~~~~~~~~^^^^^
AssertionError: <class 'test.test_ordered_dict.OrderedDictTests.test_reference_loop.<locals>.A'> is not None

======================================================================
FAIL: test_reference_loop (test.test_ordered_dict.PurePythonOrderedDictTests.test_reference_loop)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_ordered_dict.py", line 679, in test_reference_loop
    self.assertIsNone(r())
    ~~~~~~~~~~~~~~~~~^^^^^
AssertionError: <class 'test.test_ordered_dict.OrderedDictTests.test_reference_loop.<locals>.A'> is not None

----------------------------------------------------------------------
Ran 283 tests in 3.262s

FAILED (failures=4)
test test_ordered_dict failed
0:01:21 load avg: 3.81 [32/44/1] test_patma -- test_ordered_dict failed (4 failures)
0:01:21 load avg: 3.81 [33/44/1] test_pickle
0:01:33 load avg: 3.63 [34/44/1] test_pprint
0:01:34 load avg: 3.63 [35/44/1] test_re
0:01:36 load avg: 3.63 [36/44/1] test_set
0:01:45 load avg: 3.68 [37/44/1] test_sqlite3
test_sqlite3 skipped -- No module named '_sqlite3'
0:01:46 load avg: 3.68 [38/44/1] test_statistics -- test_sqlite3 skipped
0:01:57 load avg: 3.50 [39/44/1] test_str
0:02:06 load avg: 3.19 [40/44/1] test_struct
test_1530559 (test.test_struct.StructTest.test_1530559) ... ok
test_705836 (test.test_struct.StructTest.test_705836) ... ok
test_Struct_reinitialization (test.test_struct.StructTest.test_Struct_reinitialization) ... ok
test__sizeof__ (test.test_struct.StructTest.test__sizeof__) ... ok
test__struct_reference_cycle_cleaned_up (test.test_struct.StructTest.test__struct_reference_cycle_cleaned_up) ... FAIL
test__struct_types_immutable (test.test_struct.StructTest.test__struct_types_immutable) ... ok
test_bool (test.test_struct.StructTest.test_bool) ... ok
test_boundary_error_message (test.test_struct.StructTest.test_boundary_error_message) ... ok
test_boundary_error_message_with_large_offset (test.test_struct.StructTest.test_boundary_error_message_with_large_offset) ... ok
test_boundary_error_message_with_negative_offset (test.test_struct.StructTest.test_boundary_error_message_with_negative_offset) ... ok
test_calcsize (test.test_struct.StructTest.test_calcsize) ... ok
test_consistence (test.test_struct.StructTest.test_consistence) ... ok
test_count_overflow (test.test_struct.StructTest.test_count_overflow) ... ok
test_format_attr (test.test_struct.StructTest.test_format_attr) ... ok
test_integers (test.test_struct.StructTest.test_integers) ... ok
test_isbigendian (test.test_struct.StructTest.test_isbigendian) ... ok
test_issue29802 (test.test_struct.StructTest.test_issue29802) ... ok
test_issue35714 (test.test_struct.StructTest.test_issue35714) ... ok
test_issue98248 (test.test_struct.StructTest.test_issue98248) ... ok
test_issue98248_error_propagation (test.test_struct.StructTest.test_issue98248_error_propagation) ... ok
test_nN_code (test.test_struct.StructTest.test_nN_code) ... ok
test_new_features (test.test_struct.StructTest.test_new_features) ... ok
test_p_code (test.test_struct.StructTest.test_p_code) ... ok
test_pack_into (test.test_struct.StructTest.test_pack_into) ... ok
test_pack_into_fn (test.test_struct.StructTest.test_pack_into_fn) ... ok
test_repr (test.test_struct.StructTest.test_repr) ... ok
test_struct_cleans_up_at_runtime_shutdown (test.test_struct.StructTest.test_struct_cleans_up_at_runtime_shutdown) ... ok
test_struct_subclass_instantiation (test.test_struct.StructTest.test_struct_subclass_instantiation) ... ok
test_trailing_counter (test.test_struct.StructTest.test_trailing_counter) ... ok
test_transitiveness (test.test_struct.StructTest.test_transitiveness) ... ok
test_unpack_from (test.test_struct.StructTest.test_unpack_from) ... ok
test_unpack_with_buffer (test.test_struct.StructTest.test_unpack_with_buffer) ... ok
test_arbitrary_buffer (test.test_struct.UnpackIteratorTest.test_arbitrary_buffer) ... ok
test_construct (test.test_struct.UnpackIteratorTest.test_construct) ... ok
test_half_float (test.test_struct.UnpackIteratorTest.test_half_float) ... ok
test_iterate (test.test_struct.UnpackIteratorTest.test_iterate) ... ok
test_length_hint (test.test_struct.UnpackIteratorTest.test_length_hint) ... ok
test_module_func (test.test_struct.UnpackIteratorTest.test_module_func) ... ok
test_uninstantiable (test.test_struct.UnpackIteratorTest.test_uninstantiable) ... ok

======================================================================
FAIL: test__struct_reference_cycle_cleaned_up (test.test_struct.StructTest.test__struct_reference_cycle_cleaned_up)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_struct.py", line 689, in test__struct_reference_cycle_cleaned_up
    self.assertIsNone(
    ~~~~~~~~~~~~~~~~~^
        module_ref(), "_struct module was not garbage collected")
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: <module '_struct' from '.../cpython/build/lib.linux-x86_64-3.13/_struct.cpython-313t-x86_64-linux-gnu.so'> is not None : _struct module was not garbage collected

----------------------------------------------------------------------
Ran 39 tests in 1.714s

FAILED (failures=1)
test test_struct failed
0:02:10 load avg: 3.33 [41/44/2] test_tabnanny -- test_struct failed (1 failure)
0:02:14 load avg: 3.39 [42/44/2] test_time
0:02:20 load avg: 3.84 [43/44/2] test_xml_etree
0:02:32 load avg: 3.94 [44/44/2] test_xml_etree_c

== Tests result: FAILURE ==

1 test skipped:
    test_sqlite3

2 tests failed:
    test_ordered_dict test_struct

41 tests OK.

Total duration: 2 min 41 sec
Total tests: run=8,756 failures=5 skipped=178
Total test files: run=44/44 failed=2 skipped=1
Result: FAILURE

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs
@hroncok hroncok added the type-bug An unexpected behavior, bug, or error label May 9, 2024
@hroncok
Copy link
Contributor Author

hroncok commented May 9, 2024

All of thew failures seem to be related to reference loops/cycles.

@Eclips4 Eclips4 added topic-free-threading tests Tests in the Lib/test dir labels May 9, 2024
@colesbury
Copy link
Contributor

I think this is related to the immortalization of objects in the free-threaded build: #117783

@colesbury colesbury self-assigned this May 9, 2024
@hroncok
Copy link
Contributor Author

hroncok commented May 9, 2024

Runnin all the tests sequentially, I see even more failures.

$ ./python -m test -W
== CPython 3.13.0b1+ (heads/3.13:a98e44a5bf9, May 9 2024, 17:23:05) [GCC 13.2.1 20231205 (Red Hat 13.2.1-6)]
== Linux-6.6.13-200.fc39.x86_64-x86_64-with-glibc2.38 little-endian
== Python build: free_threading release PGO
== cwd: .../cpython/build/test_python_worker_793507æ
== CPU count: 8
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 2166996457


======================================================================
FAIL: test_cache_does_not_cause_classes_to_persist (test.test_inspect.test_inspect.TestGetattrStatic.test_cache_does_not_cause_classes_to_persist)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_inspect/test_inspect.py", line 2446, in test_cache_does_not_cause_classes_to_persist
    self.assertIsNone(weakref_to_class())
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
AssertionError: <class 'test.test_inspect.test_inspect.TestGetattrStatic.test_cache_does_not_cause_classes_to_persist.<locals>.Foo'> is not None

----------------------------------------------------------------------
Ran 336 tests in 0.567s

======================================================================
ERROR: test_builtin_with_more_than_four_children (test.test_pydoc.test_pydoc.PydocDocTest.test_builtin_with_more_than_four_children)
Tests help on builtin object which have more than four child classes.
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_pydoc/test_pydoc.py", line 537, in test_builtin_with_more_than_four_children
    text = doc.docclass(object)
  File ".../cpython/Lib/pydoc.py", line 1424, in docclass
    subclasses = sorted(
        (str(cls.__name__) for cls in type.__subclasses__(object)
         if not cls.__name__.startswith("_") and cls.__module__ == "builtins"),
        key=str.lower
    )
  File ".../cpython/Lib/pydoc.py", line 1426, in <genexpr>
    if not cls.__name__.startswith("_") and cls.__module__ == "builtins"),
                                            ^^^^^^^^^^^^^^
  File ".../cpython/Lib/test/test_inspect/test_inspect.py", line 775, in __module__
    raise AttributeError
AttributeError: . Did you mean: '__reduce__'?

----------------------------------------------------------------------
Ran 108 tests in 1.719s


======================================================================
FAIL: test_subclass_of_heap_gc_ctype_with_del_modifying_dunder_class_only_decrefs_once (test.test_capi.test_misc.CAPITest.test_subclass_of_heap_gc_ctype_with_del_modifying_dunder_class_only_decrefs_once)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_capi/test_misc.py", line 532, in test_subclass_of_heap_gc_ctype_with_del_modifying_dunder_class_only_decrefs_once
    self.assertEqual(type_refcnt - 1, B.refcnt_in_del)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 4294967294 != 4294967295

======================================================================
FAIL: test_subclass_of_heap_gc_ctype_with_tpdealloc_decrefs_once (test.test_capi.test_misc.CAPITest.test_subclass_of_heap_gc_ctype_with_tpdealloc_decrefs_once)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_capi/test_misc.py", line 499, in test_subclass_of_heap_gc_ctype_with_tpdealloc_decrefs_once
    self.assertEqual(type_refcnt - 1, sys.getrefcount(HeapGcCTypeSubclass))
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 4294967294 != 4294967295

----------------------------------------------------------------------
Ran 888 tests in 7.437s

======================================================================
FAIL: test_remove_subclass (test.test_descr.ClassPropertiesAndMethods.test_remove_subclass)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_descr.py", line 5032, in test_remove_subclass
    self.assertEqual(Parent.__subclasses__(), [])
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Lists differ: [<class 'test.test_descr.ClassPropertiesAn[42 chars]ld'>] != []

First list contains 1 additional elements.
First extra element 0:
<class 'test.test_descr.ClassPropertiesAndMethods.test_remove_subclass.<locals>.Child'>

- [<class 'test.test_descr.ClassPropertiesAndMethods.test_remove_subclass.<locals>.Child'>]
+ []

----------------------------------------------------------------------
Ran 156 tests in 3.112s

======================================================================
FAIL: test_class (test.test_gc.GCTests.test_class)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_gc.py", line 118, in test_class
    self.assertNotEqual(gc.collect(), 0)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
AssertionError: 0 == 0

======================================================================
FAIL: test_function (test.test_gc.GCTests.test_function)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_gc.py", line 231, in test_function
    self.assertIn(gc.collect(), (2, 3))
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 0 not found in (2, 3)

======================================================================
FAIL: test_is_tracked (test.test_gc.GCTests.test_is_tracked)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_gc.py", line 601, in test_is_tracked
    self.assertTrue(gc.is_tracked(gc))
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
AssertionError: False is not true

======================================================================
FAIL: test_newinstance (test.test_gc.GCTests.test_newinstance)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_gc.py", line 154, in test_newinstance
    self.assertNotEqual(gc.collect(), 0)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
AssertionError: 0 == 0

======================================================================
FAIL: test_newstyleclass (test.test_gc.GCTests.test_newstyleclass)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_gc.py", line 125, in test_newstyleclass
    self.assertNotEqual(gc.collect(), 0)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
AssertionError: 0 == 0

======================================================================
FAIL: test_bug1055820c (test.test_gc.GCTogglingTests.test_bug1055820c)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_gc.py", line 1353, in test_bug1055820c
    self.fail("gc didn't happen after 10000 iterations")
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: gc didn't happen after 10000 iterations

======================================================================
FAIL: test_bug1055820d (test.test_gc.GCTogglingTests.test_bug1055820d)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/contextlib.py", line 85, in inner
    return func(*args, **kwds)
  File ".../cpython/Lib/test/test_gc.py", line 1421, in test_bug1055820d
    self.fail("gc didn't happen after 10000 iterations")
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: gc didn't happen after 10000 iterations

----------------------------------------------------------------------
Ran 49 tests in 13.545s

======================================================================
FAIL: test_free_after_iterating (test.test_list.ListTest.test_free_after_iterating)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/seq_tests.py", line 430, in test_free_after_iterating
    support.check_free_after_iterating(self, iter, self.type2test)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../cpython/Lib/test/support/__init__.py", line 1830, in check_free_after_iterating
    test.assertTrue(done)
    ~~~~~~~~~~~~~~~^^^^^^
AssertionError: False is not true

----------------------------------------------------------------------
Ran 59 tests in 0.669s

======================================================================
FAIL: test_clear_dict_in_ref_cycle (test.test_module.ModuleTests.test_clear_dict_in_ref_cycle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_module/__init__.py", line 119, in test_clear_dict_in_ref_cycle
    self.assertEqual(destroyed, [1])
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
AssertionError: Lists differ: [] != [1]

Second list contains 1 additional elements.
First extra element 0:
1

- []
+ [1]
?  +


======================================================================
FAIL: test_weakref (test.test_module.ModuleTests.test_weakref)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_module/__init__.py", line 127, in test_weakref
    self.assertIs(wr(), None)
    ~~~~~~~~~~~~~^^^^^^^^^^^^
AssertionError: <module 'foo'> is not None

----------------------------------------------------------------------
Ran 39 tests in 1.100s

======================================================================
FAIL: test_reference_loop (test.test_ordered_dict.CPythonOrderedDictSubclassTests.test_reference_loop)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_ordered_dict.py", line 679, in test_reference_loop
    self.assertIsNone(r())
    ~~~~~~~~~~~~~~~~~^^^^^
AssertionError: <class 'test.test_ordered_dict.OrderedDictTests.test_reference_loop.<locals>.A'> is not None

======================================================================
FAIL: test_reference_loop (test.test_ordered_dict.CPythonOrderedDictTests.test_reference_loop)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_ordered_dict.py", line 679, in test_reference_loop
    self.assertIsNone(r())
    ~~~~~~~~~~~~~~~~~^^^^^
AssertionError: <class 'test.test_ordered_dict.OrderedDictTests.test_reference_loop.<locals>.A'> is not None

======================================================================
FAIL: test_reference_loop (test.test_ordered_dict.PurePythonOrderedDictSubclassTests.test_reference_loop)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_ordered_dict.py", line 679, in test_reference_loop
    self.assertIsNone(r())
    ~~~~~~~~~~~~~~~~~^^^^^
AssertionError: <class 'test.test_ordered_dict.OrderedDictTests.test_reference_loop.<locals>.A'> is not None

======================================================================
FAIL: test_reference_loop (test.test_ordered_dict.PurePythonOrderedDictTests.test_reference_loop)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_ordered_dict.py", line 679, in test_reference_loop
    self.assertIsNone(r())
    ~~~~~~~~~~~~~~~~~^^^^^
AssertionError: <class 'test.test_ordered_dict.OrderedDictTests.test_reference_loop.<locals>.A'> is not None

----------------------------------------------------------------------
Ran 283 tests in 13.000s

======================================================================
FAIL: test__struct_reference_cycle_cleaned_up (test.test_struct.StructTest.test__struct_reference_cycle_cleaned_up)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_struct.py", line 689, in test__struct_reference_cycle_cleaned_up
    self.assertIsNone(
    ~~~~~~~~~~~~~~~~~^
        module_ref(), "_struct module was not garbage collected")
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: <module '_struct' from '.../cpython/build/lib.linux-x86_64-3.13/_struct.cpython-313t-x86_64-linux-gnu.so'> is not None : _struct module was not garbage collected

----------------------------------------------------------------------
Ran 39 tests in 0.350s

======================================================================
FAIL: test_loop_caller_importing (test.test_trace.TestCallers.test_loop_caller_importing)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_trace.py", line 352, in test_loop_caller_importing
    self.assertEqual(self.tracer.results().callers, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: {(('/[61 chars]e', 'runfunc'), ('/home/churchyard/Dokumenty/R[986 chars]): 1} != {(('/[61 chars]e', 'Trace.runfunc'), ('/home/churchyard/Dokum[992 chars]): 1}
Diff is 1409 characters long. Set self.maxDiff to None to see it.

======================================================================
FAIL: test_inst_method_calling (test.test_trace.TestFuncs.test_inst_method_calling)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_trace.py", line 313, in test_inst_method_calling
    self.assertEqual(self.tracer.results().calledfuncs, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: {('/h[75 chars]e', 'inst_method_calling'): 1, ('/home/churchy[197 chars]): 1} != {('/h[75 chars]e', 'TracedClass.inst_method_calling'): 1, ('/[221 chars]): 1}
Diff is 788 characters long. Set self.maxDiff to None to see it.

----------------------------------------------------------------------
Ran 27 tests in 3.653s

======================================================================
FAIL: test_gc_tracked (test.test_zoneinfo.test_zoneinfo.ExtensionBuiltTest.test_gc_tracked)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../cpython/Lib/test/test_zoneinfo/test_zoneinfo.py", line 1937, in test_gc_tracked
    self.assertTrue(gc.is_tracked(py_zoneinfo.ZoneInfo))
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: False is not true

----------------------------------------------------------------------
Ran 239 tests in 0.278s

== Tests result: FAILURE ==

28 tests skipped:
    test.test_asyncio.test_windows_events
    test.test_asyncio.test_windows_utils test.test_gdb.test_backtrace
    test.test_gdb.test_cfunction test.test_gdb.test_cfunction_full
    test.test_gdb.test_misc test.test_gdb.test_pretty_print
    test_android test_dbm_gnu test_dbm_ndbm test_dbm_sqlite3
    test_devpoll test_idle test_kqueue test_launcher test_msvcrt
    test_readline test_sqlite3 test_startfile test_tcl test_tkinter
    test_ttk test_ttk_textonly test_turtle test_winapi
    test_winconsoleio test_winreg test_wmi

9 tests skipped (resource denied):
    test_curses test_peg_generator test_pyrepl test_smtpnet
    test_socketserver test_urllib2net test_urllibnet test_winsound
    test_zipfile64

1 test run no tests:
    test_interpreters

11 tests failed:
    test.test_inspect.test_inspect test.test_pydoc.test_pydoc
    test_capi test_descr test_gc test_list test_module
    test_ordered_dict test_struct test_trace test_zoneinfo

428 tests OK.

Total duration: 55 min 13 sec
Total tests: run=42,270 failures=22 skipped=1,794
Total test files: run=468/477 failed=11 skipped=28 resource_denied=9 run_no_tests=1
Result: FAILURE

They succeed when run with -j4.

colesbury added a commit to colesbury/cpython that referenced this issue May 9, 2024
@colesbury
pythongh-118846: Fix PGO tests in free-threaded build
9f8175b 
Avoid immortalizing objects in some tests that test behavior of
collecting classes or modules.

This fixes test_ordered_dict and test_struct.
colesbury added a commit to colesbury/cpython that referenced this issue May 9, 2024
@colesbury
pythongh-118846: Fix PGO tests in free-threaded build
f203ebf 
Avoid immortalizing objects in some tests that test behavior of
collecting classes or modules.

This fixes test_ordered_dict and test_struct.
@bedevere-app bedevere-app bot mentioned this issue May 9, 2024
Merged
colesbury added a commit to colesbury/cpython that referenced this issue May 9, 2024
@colesbury
pythongh-118846: Fix free-threading test failures when run sequentially
852175f 
The free-threaded build currently immortalizes some objects once the
first thread is started. This can lead to test failures depending on the
order in which tests are run. This PR addresses those failures by
suppressing immortalization or skipping the affected tests.
@bedevere-app bedevere-app bot mentioned this issue May 9, 2024
Merged
colesbury added a commit that referenced this issue May 9, 2024
@colesbury
gh-118846: Fix PGO tests in free-threaded build (#118862)
1b1db2f 
Avoid immortalizing objects in tests that verify garbage collection of
classes or modules.

This fixes test_ordered_dict and test_struct.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 9, 2024
@colesbury @miss-islington
pythongh-118846: Fix PGO tests in free-threaded build (pythonGH-118862)
89b8e85 
Avoid immortalizing objects in tests that verify garbage collection of
classes or modules.

This fixes test_ordered_dict and test_struct.
(cherry picked from commit 1b1db2f)

Co-authored-by: Sam Gross <colesbury@gmail.com>
@bedevere-app bedevere-app bot mentioned this issue May 9, 2024
Merged
colesbury added a commit that referenced this issue May 9, 2024
@miss-islington @colesbury
[3.13] gh-118846: Fix PGO tests in free-threaded build (GH-118862) (#…
d4c6948 
…118867)

Avoid immortalizing objects in tests that verify garbage collection of
classes or modules.

This fixes test_ordered_dict and test_struct.
(cherry picked from commit 1b1db2f)

Co-authored-by: Sam Gross <colesbury@gmail.com>
SonicField pushed a commit to SonicField/cpython that referenced this issue May 10, 2024
@colesbury @SonicField
pythongh-118846: Fix PGO tests in free-threaded build (python#118862)
888d3d7 
Avoid immortalizing objects in tests that verify garbage collection of
classes or modules.

This fixes test_ordered_dict and test_struct.
hroncok pushed a commit to fedora-python/cpython that referenced this issue May 10, 2024
@colesbury @hroncok
00428: pythongh-118846: Fix PGO tests in free-threaded build
f2440de 
Avoid immortalizing objects in tests that verify garbage collection of
classes or modules.

This fixes test_ordered_dict and test_struct.

(cherry picked from commit 1b1db2f)
colesbury added a commit to colesbury/cpython that referenced this issue May 10, 2024
@colesbury
Merge branch 'main' into pythongh-118846-all-tests
923dd2d
@impredicative
Copy link

I was at least able to install it when using --disable-gil without --enable-optimizations, but various errors may still exist even if they were not a blocker.

colesbury added a commit that referenced this issue May 10, 2024
@colesbury
gh-118846: Fix free-threading test failures when run sequentially (#1…
b309c8e 
…18864)

The free-threaded build currently immortalizes some objects once the
first thread is started. This can lead to test failures depending on the
order in which tests are run. This PR addresses those failures by
suppressing immortalization or skipping the affected tests.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 10, 2024
@colesbury @miss-islington
pythongh-118846: Fix free-threading test failures when run sequential…
305f67e 
…ly (pythonGH-118864)

The free-threaded build currently immortalizes some objects once the
first thread is started. This can lead to test failures depending on the
order in which tests are run. This PR addresses those failures by
suppressing immortalization or skipping the affected tests.
(cherry picked from commit b309c8e)

Co-authored-by: Sam Gross <colesbury@gmail.com>
@bedevere-app bedevere-app bot mentioned this issue May 10, 2024
Merged
colesbury added a commit that referenced this issue May 10, 2024
@miss-islington @colesbury
[3.13] gh-118846: Fix free-threading test failures when run sequentia…
cceb758 
…lly (GH-118864) (#118927)

The free-threaded build currently immortalizes some objects once the
first thread is started. This can lead to test failures depending on the
order in which tests are run. This PR addresses those failures by
suppressing immortalization or skipping the affected tests.
(cherry picked from commit b309c8e)

Co-authored-by: Sam Gross <colesbury@gmail.com>
@hroncok hroncok closed this as completed May 11, 2024
@JelleZijlstra JelleZijlstra mentioned this issue May 28, 2024
Closed
estyxx pushed a commit to estyxx/cpython that referenced this issue Jul 17, 2024
@colesbury @estyxx
pythongh-118846: Fix free-threading test failures when run sequential…
b1970ec 
…ly (python#118864)

The free-threaded build currently immortalizes some objects once the
first thread is started. This can lead to test failures depending on the
order in which tests are run. This PR addresses those failures by
suppressing immortalization or skipping the affected tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@colesbury colesbury

Labels
tests Tests in the Lib/test dir topic-free-threading type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@impredicative @colesbury @hroncok @Eclips4