Failure in CI - CalledProcessError for command 'which brew'

[jaraco]

Sometime in the past 5 days, the tests have started failing during collection with:

________________________ ERROR collecting docs/conf.py _________________________
<frozen importlib.util>:95: in find_spec
    ???
jaraco/xonsh.xsh:399: in <module>
    if !(which brew):
.tox/py/lib/python3.11/site-packages/xonsh/procs/pipelines.py:192: in __bool__
    return self.returncode == 0
.tox/py/lib/python3.11/site-packages/xonsh/procs/pipelines.py:674: in returncode
    self.end()
.tox/py/lib/python3.11/site-packages/xonsh/procs/pipelines.py:459: in end
    self._end(tee_output=tee_output)
.tox/py/lib/python3.11/site-packages/xonsh/procs/pipelines.py:467: in _end
    for _ in self.tee_stdout():
.tox/py/lib/python3.11/site-packages/xonsh/procs/pipelines.py:369: in tee_stdout
    for line in self.iterraw():
.tox/py/lib/python3.11/site-packages/xonsh/procs/pipelines.py:338: in iterraw
    self.end(tee_output=False)
.tox/py/lib/python3.11/site-packages/xonsh/procs/pipelines.py:459: in end
    self._end(tee_output=tee_output)
.tox/py/lib/python3.11/site-packages/xonsh/procs/pipelines.py:478: in _end
    self._raise_subproc_error()
.tox/py/lib/python3.11/site-packages/xonsh/procs/pipelines.py:603: in _raise_subproc_error
    raise subprocess.CalledProcessError(rtn, spec.args, output=self.output)
E   subprocess.CalledProcessError: Command '['which', 'brew']' returned non-zero exit status 1.

However, running the tests locally pass just fine (on macOS and Linux). I don't get it. Why is a CalledProcessError being raised when the code is clearly checking the returncode.

[jaraco]

Some dependencies changed between the successful and failing builds, but nothing seems particularly relevant. Plus, why can't I replicate the issue on Ubuntu docker images?

[jaraco]

I did observe that xonsh has a pytest plugin, and part of this plugin triggers xonsh setup, and that function does appear to set the environment to raise on error.

Looking at my local test runs and the ones that passed earlier, I note that the jaraco/xonsh.xsh file doesn't get tested.

But wait, the import of that module is happening during collection of docs/conf.py. How does that make any sense?

[jaraco]

Pinning to pytest<8.1 works around the issue. Same for pytest<8.2.

[jaraco]

I am able to replicate the failure locally using act to replicate the GHA environment:

 jaraco.xonsh main @ act --job test --matrix python:3.12 --matrix platform:ubuntu-latest --container-architecture linux/amd64

[jaraco]

Digging into the failure, I can see that at bottom of the stack, in importlib.util, the context is:

/usr/lib/python3.10/importlib/util.py:94: in find_spec
    parent = __import__(parent_name, fromlist=['__path__'])

At the point where that's called, parent_name is jaraco.xonsh.docs. It seems that something about pytest's collection is attempting to import the docs directory from jaraco.xonsh, which really doesn't make a whole lot of sense, given the tree:

 jaraco.xonsh main @ tree
.
├── LICENSE
├── NEWS.rst
├── README.rst
├── docs
│   ├── conf.py
│   ├── history.rst
│   └── index.rst
├── jaraco
│   └── xonsh.xsh
├── mypy.ini
├── pyproject.toml
├── pytest.ini
├── ruff.toml
├── towncrier.toml
└── tox.ini

jaraco.xonsh is not a package and docs is a sibling of jaraco.

[jaraco]

Disabling the xonsh plugin (-p no:xonsh) suppresses the failure.