test_pyexpat: test_exception() fails on Ubuntu and macOS jobs on GitHub Actions

[vstinner]

FAIL: test_exception (test.test_pyexpat.HandlerExceptionTest.test_exception)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_pyexpat.py", line 452, in test_exception
    parser.Parse(b"<a><b><c/></b></a>", True)
  File "../cpython-ro-srcdir/Modules/pyexpat.c", line 421, in StartElement
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_pyexpat.py", line 442, in StartElementHandler
    raise RuntimeError(name)
RuntimeError: a

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_pyexpat.py", line 472, in test_exception
    self.assertIn('call_with_frame("StartElement"', entries[1][3])
AssertionError: 'call_with_frame("StartElement"' not found in ''

• Ubuntu logs: https://github.com/python/cpython/actions/runs/6138224249/job/16654899227?pr=109229
• macOS logs: https://github.com/python/cpython/actions/runs/6138224249/job/16654899199?pr=109229

Comparison of Ubuntu logs:

• before (ok): https://github.com/python/cpython/actions/runs/6134648048/job/16647654038

  • commit 4303918
  • configure: checking for --with-system-expat... no
  • test.pythoninfo: expat.EXPAT_VERSION: expat_2.5.0
  • Tests: Result: SUCCESS

• after (error): https://github.com/python/cpython/actions/runs/6138224249/job/16654899227?pr=109229 -- PR

  • commit 28ddef7 -- PR gh-109162: libregrtest: add worker.py #109229
  • configure: checking for --with-system-expat... no
  • test.pythoninfo: expat.EXPAT_VERSION: expat_2.5.0
  • Test: 1 test failed: test_pyexpat

Linked PRs

• gh-109230: test_pyexpat no longer depends on the current directory #109233
• [3.12] gh-109230: test_pyexpat no longer depends on the current directory (GH-109233) #109241
• [3.11] gh-109230: test_pyexpat no longer depends on the current directory (GH-109233) #109242

[vstinner]

Oh wait. Maybe it's a real bug in my PR #109229.

[vstinner]

Oh. This test depends on the current working directory!

If it's run from the source directory, it pass.

$ pwd
/home/vstinner/python/main

$ ls Modules/pyexpat.c
Modules/pyexpat.c

vstinner@mona$ ./python -m test -v test_pyexpat -m test_exception  -q
(...)
Result: SUCCESS

If it's run from a different directory, it fails!

$ cd ..

$ pwd
/home/vstinner/python

$ ls Modules/pyexpat.c
ls: cannot access 'Modules/pyexpat.c': No such file or directory

$ main/python -m test -v test_pyexpat -m test_exception  -q
(...)
test test_pyexpat failed -- Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/test/test_pyexpat.py", line 452, in test_exception
    parser.Parse(b"<a><b><c/></b></a>", True)
  File "./Modules/pyexpat.c", line 421, in StartElement
  File "/home/vstinner/python/main/Lib/test/test_pyexpat.py", line 442, in StartElementHandler
    raise RuntimeError(name)
RuntimeError: a

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/test/test_pyexpat.py", line 472, in test_exception
    self.assertIn('call_with_frame("StartElement"', entries[1][3])
AssertionError: 'call_with_frame("StartElement"' not found in ''

(...)
Result: FAILURE

[vstinner]

The test checks that the traceback module is able to read a line of the Python C code in Modules/pyexpat.c. The test was added by commit 0ddbf47 in issue #66652 which added _PyTraceback_Add().

Quickly, the test was skipped if sysconfig.is_python_build() is false: commit 2b3b95b.

In 2022, the test was skipped on WASM and Win32 ARM: commit 96b344c. These two platforms build Python with cross-compilation: the target Python doesn't have access to Python C source code. A minor follow-up also skipped the test on WASI: commit 9b50585.

IMO the condition to decide to skip the test or not is weak. And the test should not depend on which directory tests are run.

[vstinner]

Oh wait. Maybe it's a real bug in my PR #109229.

My PR makes the issue easier to reproduce since running the Python test suite with -j1 option changes the current working directory and so trigger the bug.