Display path to corrupted files when pip fails to load wheel

[gaborbernat]

What's the problem this feature will solve?

This would help indentify the package that causes the problem.

Describe the solution you'd like

Currently when this happens one gets a stack trace of:

[2025-01-07T09:07:01.934Z] Successfully built pyspark

[2025-01-07T09:07:01.934Z] Installing collected packages: wcwidth, pytz, py4j, wrapt, urllib3, six, pyspark, propcache, prettytable, pluggy, packaging, numpy, multidict, jmespath, iniconfig, idna, fsspec, frozenlist, et-xmlfile, coverage, attrs, aioitertools, aiohappyeyeballs, yarl, python-dateutil, pytest, openpyxl, covdefaults, chispa,ai.remoteio, aiosignal, pytest-cov, pandas, botocore, aiohttp, aiobotocore, s3fs
[2025-01-07T09:07:01.934Z] test: exit 2 (32.29 seconds) /w> python -I -m pip install ai.remoteio chispa covdefaults numpy==1.26.4 openpyxl 'pandas<2' pyspark==3.5.0 pytest pytest-cov s3fs pid=108
[2025-01-07T09:07:01.934Z] ERROR: Exception:
[2025-01-07T09:07:01.934Z] Traceback (most recent call last):
[2025-01-07T09:07:01.934Z]   File "/w/.tox/test/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
[2025-01-07T09:07:01.934Z]     status = _inner_run()
[2025-01-07T09:07:01.934Z]              ^^^^^^^^^^^^
[2025-01-07T09:07:01.934Z]   File "/w/.tox/test/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run
[2025-01-07T09:07:01.934Z]     return self.run(options, args)
[2025-01-07T09:07:01.934Z]            ^^^^^^^^^^^^^^^^^^^^^^^
[2025-01-07T09:07:01.934Z]   File "/w/.tox/test/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 67, in wrapper
[2025-01-07T09:07:01.934Z]     return func(self, options, args)
[2025-01-07T09:07:01.934Z]            ^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-01-07T09:07:01.934Z]   File "/w/.tox/test/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 457, in run
[2025-01-07T09:07:01.934Z]     installed = install_given_reqs(
[2025-01-07T09:07:01.934Z]                 ^^^^^^^^^^^^^^^^^^^
[2025-01-07T09:07:01.934Z]   File "/w/.tox/test/lib/python3.11/site-packages/pip/_internal/req/__init__.py", line 70, in install_given_reqs
[2025-01-07T09:07:01.934Z]     requirement.install(
[2025-01-07T09:07:01.934Z]   File "/w/.tox/test/lib/python3.11/site-packages/pip/_internal/req/req_install.py", line 867, in install
[2025-01-07T09:07:01.934Z]     install_wheel(
[2025-01-07T09:07:01.934Z]   File "/w/.tox/test/lib/python3.11/site-packages/pip/_internal/operations/install/wheel.py", line 732, in install_wheel
[2025-01-07T09:07:01.934Z]     _install_wheel(
[2025-01-07T09:07:01.934Z]   File "/w/.tox/test/lib/python3.11/site-packages/pip/_internal/operations/install/wheel.py", line 590, in _install_wheel
[2025-01-07T09:07:01.934Z]     file.save()
[2025-01-07T09:07:01.934Z]   File "/w/.tox/test/lib/python3.11/site-packages/pip/_internal/operations/install/wheel.py", line 380, in save
[2025-01-07T09:07:01.934Z]     shutil.copyfileobj(f, dest, blocksize)
[2025-01-07T09:07:01.934Z]   File "/opt/bb/lib/python3.11/shutil.py", line 197, in copyfileobj
[2025-01-07T09:07:01.934Z]     buf = fsrc_read(length)
[2025-01-07T09:07:01.934Z]           ^^^^^^^^^^^^^^^^^
[2025-01-07T09:07:01.934Z]   File "/opt/bb/lib/python3.11/zipfile.py", line 966, in read
[2025-01-07T09:07:01.934Z]     data = self._read1(n)
[2025-01-07T09:07:01.934Z]            ^^^^^^^^^^^^^^
[2025-01-07T09:07:01.934Z]   File "/opt/bb/lib/python3.11/zipfile.py", line 1034, in _read1
[2025-01-07T09:07:01.934Z]     data += self._read2(n - len(data))
[2025-01-07T09:07:01.934Z]             ^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-01-07T09:07:01.934Z]   File "/opt/bb/lib/python3.11/zipfile.py", line 1069, in _read2
[2025-01-07T09:07:01.934Z]     raise EOFError
[2025-01-07T09:07:01.934Z] EOFError

From this is not clear what package caused the issue.

Alternative Solutions

One can use PIP_DEBUG but that requires CI changes.

Additional context

Better error message.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pfmoore]

This seems like a reasonable quality of life improvement - a PR fixing this would be appreciated.

[sepehr-rs]

Hi @pfmoore, I'm interested in working on this, but I'm having difficulty reproducing the bug. Could you help me with this matter? I'd really appreciate your help, thank you !

[pfmoore]

Sorry, no I have no more idea than you do. @gaborbernat can you provide a self-contained reproducer for this issue?

It might be enough to catch errors around the copyfileobj call in https://github.com/pypa/pip/blob/main/src/pip/_internal/operations/install/wheel.py#L372 and report the error, giving the name of the zipfile (self._zip_file.filename), but I'd prefer to see what that looked like in an actual reproducer, just to make sure it actually helped.

I'm guessing that this only happens when there's a wheel in the package index that's corrupt?

[gaborbernat]

Yeah, I don't have a repro, but as far as I remember it happend with corrupt wheels (either from PyPI or cache or local builds).

[sepehr-rs]

Yeah, I don't have a repro, but as far as I remember it happend with corrupt wheels (either from PyPI or cache or local builds).

I tried reproducing the error with pip download and then truncating a part of the wheel, but pip gave an output like this :

(venv) sepehr@debian:/tmp/m$ pip install numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 
Processing ./numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
ERROR: Wheel 'numpy' located at /tmp/m/numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl is invalid.

[pfmoore]

While in principle I agree with the idea that we shouldn't let unhandled exceptions get displayed to the user, I don't think that "an exception could happen here, we should catch it" is enough of a reason (we'd end up with the essential logic obscured by error handling if we did that).

So I'd prefer that we wait until we can find a proper reproducer before trying to make a change here. I'm neutral on whether we keep this issue open for reference and mark it as "waiting for reproducer", or close it on the assumption that if someone hits the same issue in the future, they can open a new issue.