wheel from sdist leads to nuisance error messages in backend

[jaraco]

In pypa/setuptools-scm#997, I observed that setuptools_scm was complaining about not being able to discover files when building from a git repo. As it turns out, the issue occurred when the pep517 build infrastructure was building a wheel from the sdist (so git metadata wouldn't be expected).

Because of the way setuptools is engineered, it will necessarily involve setuptools_scm at both phases of the build, during the sdist and then again at the wheel. It's okay to see the error message during the wheel build because it no longer needs the SCM metadata because that's been embedded in the sdist during the sdist phase. In short, I don't think there is currently a way for Setuptools or setuptools_scm to know that it's doing a "wheel from sdist" and thus to disable features like "search for files using git metadata". Therefore, I don't think there's a way to get rid of the error message. Perhaps there's something that could be done, but it would require coordination between build and the backends (setuptools) and their plugins (setuptools_scm).

Originally posted by @jaraco in pypa/setuptools-scm#997 (comment)

[jaraco]

Perhaps there's something that should be added to the build infrastructure to let the backends (and their plugins) know that it's working from an sdist, so it can make assumptions about the source code.

[pfmoore]

This is something that's come up a few times recently. My first question would be, what does it mean to be "working from a sdist"?

1. Build frontend downloads sdist, unpacks it, and passes the directory to the backend.
2. User downloads sdist, unpacks it, passes the directory to the build tool, which passes it to the backend.
3. User downloads sdist, unpacks it, modifies it, then passes the directory to the build tool, which passes it to the backend.

If we only want (1) to count as "building from the sdist", then maybe we could add a new flag to the build_wheel backend flag that says "this is an unpacked sdist".

I don't see any way of detecting (2), and more specifically, I see no way of distinguishing (2) and (3).

Unfortunately, because of (3), we can't reasonably rely on anything in the source directory to mark it as a sdist - checking for PKG-INFO doesn't work, as was recently discussed here.

[bulletmark]

Note the opening comment here says "setuptools_scm was complaining" but let's be clear what the message is:

ERROR setuptools_scm._file_finders.git listing git files failed - pretending there aren't any

It is unnerving as a user to see that ERROR message reported. I spent ages trying to track down what I was doing wrong until I discovered this bug which now refers to here.

[RonnyPfannschmidt]

i consider this particular issue a setuptools problem - other build tools cache file details in the sdist (like hatch)

[rpatterson]

ERROR setuptools_scm._file_finders.git listing git files failed - pretending there aren't any

I noticed this error today and while investigating I found some circumstances that did not reproduce the issue where $ pyproject-build is still building the wheel from the sdist. Specifically, I want $ tox to run the tests against the same ./dist/*.whl that will be uploaded to PyPI but tox's support for running against a built wheel does not build that wheel from an sdist like $ pyproject-build does. So I configured tox to do so using an external package build venv:

[testenv:.pkg]
deps =
    build
    setuptools>=64
    wheel
    setuptools_scm>=8
package_glob = {work_dir}{/}.pkg{/}tmp{/}dist{/}*.whl
commands =
    pyproject-build --outdir {work_dir}{/}.pkg{/}tmp{/}dist{/} --no-isolation \
        --skip-dependency-check

Then I can reproduce the error when I let tox build the package that way. Note that the error appears twice:

$ tox run -e py311 --override testenv.package=external --pkg-only
...
.pkg: 123 W commands[0]> pyproject-build --outdir ~/src/foo-pkg/.tox/.pkg/tmp/dist/ --no-isolation --skip-dependency-check [tox/tox_env/api.py:427]
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - setuptools>=45
  - setuptools_scm>=6.2
  - wheel
* Getting build dependencies for sdist...
running egg_info
...
* Building sdist...
...
* Building wheel from sdist
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - setuptools>=45
  - setuptools_scm>=6.2
  - wheel
* Getting build dependencies for wheel...
running egg_info
...
writing top-level names to src/foo-pkg.egg-info/top_level.txt
ERROR setuptools_scm._file_finders.git listing git files failed - pretending there aren't any
reading manifest file 'src/foo-pkg.egg-info/SOURCES.txt'
writing manifest file 'src/foo-pkg.egg-info/SOURCES.txt'
* Building wheel...
...
writing top-level names to src/foo-pkg.egg-info/top_level.txt
ERROR setuptools_scm._file_finders.git listing git files failed - pretending there aren't any
reading manifest file 'src/foo-pkg.egg-info/SOURCES.txt'
writing manifest file 'src/foo-pkg.egg-info/SOURCES.txt'
...

But I can not reproduce the error when tox runs the same $ pyproject-build command in one of the test ./.tox/py3##/ venvs, as opposed to its ./.tox/.pkg/ venv: $ tox exec -e py310 -- pyproject-build --outdir ~/src/foo-pkg/.tox/.pkg/tmp/dist/ --no-isolation --skip-dependency-check. If I remove the --no-isolation option, however, it reproduces the error again. So I'm guessing that there's something about the venv types that tox's .pkg and pyproject-build's isolation use that reproduces the error.

Hope this helps.