test_embed fails when generating PGO profile information on Linux

[diegorusso]

Bug report

Bug description:

To replicate the error:

./configure --enable-optimizations
make -j

This will fail when running tests for generating the PGO profiles on Linux (x86 and AArch64)

...
...
# Next, run the profile task to generate the profile information.
./python -m test --pgo --timeout=
Using random seed: 513698620
0:00:00 load avg: 1.94 Run 44 tests sequentially in a single process
0:00:00 load avg: 1.94 [ 1/44] test_array
0:00:00 load avg: 1.94 [ 2/44] test_base64
0:00:01 load avg: 1.94 [ 3/44] test_binascii
0:00:01 load avg: 1.94 [ 4/44] test_binop
0:00:01 load avg: 1.94 [ 5/44] test_bisect
0:00:01 load avg: 1.94 [ 6/44] test_bytes
0:00:03 load avg: 1.86 [ 7/44] test_bz2
0:00:04 load avg: 1.86 [ 8/44] test_cmath
0:00:04 load avg: 1.86 [ 9/44] test_codecs
0:00:04 load avg: 1.86 [10/44] test_collections
0:00:05 load avg: 1.86 [11/44] test_complex
0:00:06 load avg: 1.86 [12/44] test_dataclasses
0:00:06 load avg: 1.86 [13/44] test_datetime
0:00:10 load avg: 1.79 [14/44] test_decimal
0:00:12 load avg: 1.73 [15/44] test_difflib
0:00:13 load avg: 1.73 [16/44] test_embed
test test_embed failed
0:00:19 load avg: 1.67 [17/44] test_float -- test_embed failed (1 failure)
0:00:19 load avg: 1.67 [18/44] test_fstring
0:00:21 load avg: 1.67 [19/44] test_functools
0:00:21 load avg: 1.67 [20/44] test_generators
0:00:22 load avg: 1.61 [21/44] test_hashlib
0:00:23 load avg: 1.61 [22/44] test_heapq
0:00:23 load avg: 1.61 [23/44] test_int
0:00:24 load avg: 1.61 [24/44] test_itertools
0:00:27 load avg: 1.61 [25/44] test_json
0:00:30 load avg: 1.57 [26/44] test_long
0:00:31 load avg: 1.57 [27/44] test_lzma
0:00:32 load avg: 1.57 [28/44] test_math
0:00:34 load avg: 1.52 [29/44] test_memoryview
0:00:34 load avg: 1.52 [30/44] test_operator
0:00:35 load avg: 1.52 [31/44] test_ordered_dict
0:00:36 load avg: 1.52 [32/44] test_patma
0:00:36 load avg: 1.52 [33/44] test_pickle
0:00:39 load avg: 1.48 [34/44] test_pprint
0:00:40 load avg: 1.48 [35/44] test_re
0:00:40 load avg: 1.48 [36/44] test_set
0:00:44 load avg: 1.44 [37/44] test_sqlite3
0:00:44 load avg: 1.44 [38/44] test_statistics
0:00:49 load avg: 1.40 [39/44] test_str
0:00:50 load avg: 1.40 [40/44] test_struct
0:00:51 load avg: 1.40 [41/44] test_tabnanny
0:00:51 load avg: 1.40 [42/44] test_time
0:00:53 load avg: 1.29 [43/44] test_xml_etree
0:00:54 load avg: 1.29 [44/44] test_xml_etree_c

Total duration: 56.2 sec
Total tests: run=9,500 failures=1 skipped=207
Total test files: run=44/44 failed=1
Result: FAILURE
make: *** [Makefile:937: profile-run-stamp] Error 2

On macOS it works without any problem.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

[vstinner]

This will fail when running tests for generating the PGO profiles on Linux (x86 and AArch64)

I cannot reproduce the issue on x86-64 (Fedora 41). I used commands:

./configure --enable-optimizations
make

Also, when I run the test manuallly, it works as expected:

./python -m test -v test_embed 

[vstinner]

When the bug occurs, please run ./python -m test -v test_embed.

[diegorusso]

That's interesting:

======================================================================
FAIL: test_init_setpythonhome (test.test_embed.InitConfigTests.test_init_setpythonhome)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/work/ce-sw/repos/cpython/Lib/test/test_embed.py", line 1462, in test_init_setpythonhome
    self.check_all_configs("test_init_setpythonhome", config,
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                           api=API_COMPAT, env=env)
                           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/work/ce-sw/repos/cpython/Lib/test/test_embed.py", line 930, in check_all_configs
    self.assertEqual(err.rstrip(), stderr)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '<frozen site>:100: RuntimeWarning: Unexpe[222 chars]thon' != ''
- <frozen site>:100: RuntimeWarning: Unexpected value in sys.prefix, expected /tmp/venv, got /home/user/work/ce-sw/repos/cpython
- <frozen site>:100: RuntimeWarning: Unexpected value in sys.exec_prefix, expected /tmp/venv, got /home/user/work/ce-sw/repos/cpython


----------------------------------------------------------------------
Ran 82 tests in 4.421s

FAILED (failures=1, skipped=5)
test test_embed failed
test_embed failed (1 failure)

== Tests result: FAILURE ==

1 test failed:
    test_embed

Total duration: 4.5 sec
Total tests: run=82 failures=1 skipped=5
Total test files: run=1/1 failed=1
Result: FAILURE

[diegorusso]

I was in a venv and the test failed!

[mdboom]

I was in a venv and the test failed!

No worries. Does that mean we can close this issue?

[diegorusso]

test_embed might fail if you run it within an existent venv. This happened when running pyperformance that the user installs in a venv. The trick is not to activate the venv and call pyperformance with the full path venv/bin/pyperformance