rootdir for pytester identified as /tmp when passing a tmpfile in /tmp as part of a argument

[BeyondEvil]

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

Please feel free to edit the title 😅

This PR made some of my tests start to fail with PermissionError: [Errno 13] Permission denied: '/tmp/snap-private-tmp/__init__.py'

https://github.com/pytest-dev/pytest-metadata/actions/runs/7365057317/job/20046008077

The tests pass against c7ee5599, but fail on this merge.

Running it locally I also get some warnings:

warnings

=============================== warnings summary ===============================
../../../../../../../../../Users/jimbrannlund/dev/pytest/pytest/src/_pytest/config/__init__.py:1211
  /Users/jimbrannlund/dev/pytest/pytest/src/_pytest/config/__init__.py:1211: PytestAssertRewriteWarning: Module already imported so cannot be rewritten: pytest_metadata
    self._mark_plugins_for_rewrite(hook)

../../../../../../../../../Users/jimbrannlund/dev/pytest/pytest/src/_pytest/config/__init__.py:1211
  /Users/jimbrannlund/dev/pytest/pytest/src/_pytest/config/__init__.py:1211: PytestAssertRewriteWarning: Module already imported so cannot be rewritten: tests
    self._mark_plugins_for_rewrite(hook)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

I tried to figure out exactly where it goes wrong, but failed. It's entirely possible there's something wrong in my plugin that was just surfaced by the changes in this PR.

Please let me know what information I can contribute with.

pip list

Package           Version                  Editable project location
----------------- ------------------------ ----------------------------------------------
attrs             21.4.0
black             22.1.0
cfgv              3.3.1
click             8.0.4
distlib           0.3.4
exceptiongroup    1.2.0
filelock          3.6.0
flake8            4.0.1
identify          2.4.12
iniconfig         1.1.1
mccabe            0.6.1
mypy-extensions   0.4.3
nodeenv           1.6.0
packaging         21.3
pathspec          0.9.0
pip               22.0.3
platformdirs      2.5.1
pluggy            1.3.0
pre-commit        2.17.0
py                1.11.0
pycodestyle       2.8.0
pyflakes          2.4.0
pyparsing         3.0.7
pytest            7.2.0.dev1110+gacd445a3f /Users/jimbrannlund/dev/pytest/pytest
pytest-metadata   2.0.4                    /Users/jimbrannlund/dev/pytest/pytest-metadata
PyYAML            6.0
setuptools        60.6.0
six               1.16.0
toml              0.10.2
tomli             2.0.1
tox               3.24.5
typing_extensions 4.1.1
virtualenv        20.14.0
wheel             0.37.1

I've tried the code on python 3.9...3.13 and it fails on all of them.

CI is Linux
Locally I'm on a Mac

[RonnyPfannschmidt]

rootdir is identified as /tmp - something went deep wrong - most likely related to using named temporary files instead of the testdir

since the metadata file is in /tmp, the folder finding gets confused it seems

[bluetech]

Right to add o what @RonnyPfannschmidt said, this is a combination of two things:

Problem

pytest bootstrapping:

pytest uses an elaborate procedure to decide what to use for the rootdir. Part of it depends on the command line arguments passed to pytest, e.g. pytest /my/dir/my_test.py might set /my/dir as the rootdir if there's nothing else which sets it.

The comic part is that this has to happen before plugins are loaded. So if the invocation is pytest --metadata-from-json /tmp/somefile.json, pytest doesn't know yet that /tmp/somefile.json belongs to the --metadata-from-json, but takes it as a test argument. And then it makes /tmp the rootdir.

This is a pretty silly but it's how it is now.

Changes in pytest 8:

The above has always been true so why does it start failing with pytest 8? The reason is that PR #11646 has made pytest scan the rootdir a bit more due to larger reliance on the "genitems" mechanism. I personally think it's fair and should not be an issue in itself, but we'll see...

Workarounds

1. Explicitly pass --rootdir=.

2. Change --metadata-from-json /tmp/somefile.json to --metadata-from-json=/tmp/somefile.json

3. Put the somefile.json in the testdir instead of using tempfile which puts it in /tmp.

Possible solutions in pytest

1. Somehow change pytest bootstrapping so it doesn't consider random arguments for rootdir

2. (Not a root cause solution) Handle permission errors and ignore the files (I don't like this one... but can be convinced)

3. (Not a root cause solution) Change collection to be more like before in avoiding trying to touch paths that we're sure will be deselected by the initial paths.

[BeyondEvil]

I've implemented workaround number 3.

Probably good hygiene to keep any created files in the temporary test directory anyway.

Let me know if there's anything I can help with.

[RonnyPfannschmidt]

Collection should be limited to the invocations directory in the case we hit here