-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Python 3.11.1 breaks setuptools builds on Fedora #3761
Comments
cc @hroncok |
Also having the same issue with ubuntu |
Thanks for the nudge. I'd forgotten about this issue and plan to look tomorrow. |
The repro instructions aren't working for me. I created this Dockerfile: from fedora:rawhide
run dnf install -y python3.11 python-launcher
run curl https://files.pythonhosted.org/packages/fe/31/2c88d37d4f8036c6648745e74ec51f73a6ba945a06e069601900f81fe00a/setuptools-67.4.0.tar.gz -o - | tar xz
run dnf install -y '/usr/lib/rpm/redhat/pyproject_buildrequires.py'
run ln -s setuptools* setuptools-src
workdir setuptools-src
run rm -r *.egg-info
cmd py -B /usr/lib/rpm/redhat/pyproject_buildrequires.py --generate-extras --python3_pkgversion 3 And running it emits an error about draft $ docker run -it @$(docker build -q .)
Import error: No module named 'packaging' Edit: Adding |
|
I tried entering a debugger where the issue occurs, but something about the
|
Looking at the traceback for the I added this statement to the Dockerfile to enable debugging:
|
Aha. So part of the issue is that when parsing the configuration, a |
I think we saw something similar in pypa/virtualenv#2493 (comment) Since the problem only occurred at test-time, we solved that by deleting the prepare_metadata_for_build_wheel-produced .dist-info directory after reading the data from it. I wonder if |
Good news. I've been able to replicate the failure without involving pyproject_buildrequires. Simply invoking from fedora:rawhide
run dnf install -y python3.11 python-launcher python-packaging
run curl https://files.pythonhosted.org/packages/fe/31/2c88d37d4f8036c6648745e74ec51f73a6ba945a06e069601900f81fe00a/setuptools-67.4.0.tar.gz -o - | tar xz
run ln -s setuptools* setuptools-src
workdir setuptools-src
run rm -r *.egg-info
cmd py setup.py egg_info This repro should make debugging easier because it (a) leaves |
The place that Lines 486 to 487 in 9022a21
When the |
How confident are you that the from fedora:rawhide
run dnf install -y python3.11 python-launcher python-packaging
run curl https://files.pythonhosted.org/packages/fe/31/2c88d37d4f8036c6648745e74ec51f73a6ba945a06e069601900f81fe00a/setuptools-67.4.0.tar.gz -o - | tar xz
run ln -s setuptools* setuptools-src
# downgrade importlib.metadata
run curl https://raw.githubusercontent.com/python/cpython/46a493e2db942cada2dd1abd468d52725ceba992/Lib/importlib/metadata/__init__.py -o /usr/lib64/python3.11/importlib/metadata/__init__.py
workdir setuptools-src
run rm -r *.egg-info
cmd py setup.py egg_info But the error still occurs. Is there a way to easily get a Python 3.11.0 install? |
|
Is still get the KeyError with Python 3.11.0. |
When Python was updated from 3.11.0 to 3.11.1 we started to see this failure. At that point, we had setuptools 65.5.1. Will try with that one. |
Even with 65.5.1 and Python 3.11.0 I still get the KeyError. It has been more than a month since we tested this with @stratakis and I am quite sure it started on 3.11.1, but I really cannot get 3.11.0 to work now :( |
I don't know what's happening here. I clearly see the point in history where this started in here: https://koschei.fedoraproject.org/package/python-setuptools?last_seen_ts=1671377122&collection=f39 but the logs are unfortunately already garbage-collected. |
Also when we updated Python 3.10 in Fedora to a version which contained those change we observed the same failure again. I bisected the issue with my own reproducer at the time and it pointed to the importlib.metadata commit, so I would say fairly confident back then. |
The importlib metadata commit fixed some bugs in metadata discovery, so potentially there was some corrupted or nonstandard metadata lying around on sys.path, maybe leaked from elsewhere. Since we’ve excluded it as a factor, I’m left thinking we should focus on the undersirable behavior directly and not as a regression. To some extent, the failure is working as intended. Setuptools expects itself to be present as a build dependency… including its metadata. That’s why there’s a bootstrap.egg-info in the github repo—to enable the first build in a clean checkout. The fact that it’s failing to find entry_points during configure parsing is just an early indication that key functionality is missing. You may have said it in the report already but I didn’t get it. Can you explain why you’re deleting the metadata? |
In Fedora, we have a policy to not ship pre-generated things -- and we always deleted this, to ensure it's generated during the build. Until recently, it worked. |
To add to this, I was testing setuptools 62.6.0 version which we have on Fedora 37 and the same issue occurs, however even by deleting the egg-info dir the problem is not solved. And of course it's the same drill, fails with Python 3.11.1 but succeeds with Python 3.11.0 |
This leaks from setuptools: pypa/setuptools#3761 Resolves: rhbz#2183387
Since Python 3.11.1, the setuptools builds are broken if the pre- generated egg-info directory is removed. We work around the issue by extracting the bootstrap.egg-info directory from the github tarball. Note that the bootstrap build doesn't work, however on F37 we won't bootstrap anyway, so this is considered a workaround. See also: pypa/setuptools#3761
* Modernize repo * Run 'make format' * Add changelog configuration * Update releasing instructions * Fix tests location and run moto on CI * Remove readonly * Run http server in background in functional test * Install kinto_http for functional tests * Enable setuptools-scm and remove useless MANIFEST.in * Downgrade to python 3.11.0 See pypa/setuptools#3761 (comment) * Use psycopg2-binary (only in tests) * Get rid of tox * Use kinto optional deps * Try without extra dev deps * Install dev deps in two steps * Try @grahamalama's fix
setuptools version
65.5.1-65.6.3
Python version
Python 3.11.1
OS
Fedora Linux
Additional environment information
This was observed in Fedora Linux as soon as we updated to 3.11.1
Description
Since python/cpython@07734a4 setuptools builds have started failing on Fedora.
It seems there are two issues I've discovered so far and bisecting the problem led me to the specific Python commit.
First when trying to build setuptools through the pypi tarball after an initial bootstrap build (meaning we delete the egg-info directory on the second build) and then our pyproject script tries to generate the build requirements we get a
KeyError: 'entry_points'
error.Secondly if I add the build requirements manually and then create the wheel through
python3 setup.py bdist_wheel
after again removing the pre-generated egg-info directory I get anerror: [Errno 2] No such file or directory: 'build/bdist.linux-x86_64/wheel/setuptools-65.6.3-py3.11.egg-info/PKG-INFO'
error.Tested it from 65.5.1 and on-wards. By removing that commit or by not removing the egg-info directory, everything works like before.
Expected behavior
Expected for the errors not to happen
How to Reproduce
For the first part:
On a Fedora 37 or Rawhide system
dnf install python3.11
which will bring you the latest Python 3.11.1 or compile CPython from the v3.11.1 tagGet the setuptools tarball from pypi and unzip it
Delete the egg-info directory
dnf install '/usr/lib/rpm/redhat/pyproject_buildrequires.py'
And inside the dir do
python3 -B /usr/lib/rpm/redhat/pyproject_buildrequires.py --generate-extras --python3_pkgversion 3
By downgrading to 3.11.0 or skipping python/cpython@07734a4 the issue disappears.
For the second part:
A reproducer which should be non-fedora specific although it shows a different error.
Compile and/or install Python 3.11.1
In a system where setuptools is already installed.
Get the pypi tarball
Unzip it and delete the egg-info directory
Install wheel
python3 setup.py bdist_wheel
Same as before, if I downgrade to 3.11.0 or skip the specific commit the issue disappears.
Output
First part:
Second part:
The text was updated successfully, but these errors were encountered: