Uploading a bad wheel breaks installs formerly relying on source distribution

[indygreg]

• Pip version: 8.1.2
• Python version: all
• Operating System: all

Description:

eliben/pycparser#148 is an interesting scenario for pip. pycparser 2.14 had a .tar.gz file on PyPI for several months. All was working well.

Earlier today, someone (I assume the maintainer of pycparser) uploaded a source wheel to PyPI. Normally this would "just work." However, the wheel appeared to be produced from a separate commit than the previous source distribution. Unfortunately, this meant that wheel-supporting pip clients everywhere started downloading the "bad" wheel (because they prefer wheels over .tar.gz) and random things like cffi, the cryptography package, and even Let's Encrypt clients started breaking.

OK, so a mistake was made in packaging. Garbage in, garbage out. Can't blame pip/PyPI for that.

However, many people pin hashes in their requirements file (yay determinism and security). Once the wheel was uploaded to PyPI, pip clients pinning hashes (like Let's Encrypt) started complaining about hash mismatch. Why? Because pip was downloading the .whl and only the .whl. Even though a .tar.gz with the pinned hash still existed and was available to download (and would be downloaded if pip --no-binary pycrypto was used), pip didn't do that. Instead, you get a scary message making it sound like files have been tampered with:

THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    pycparser==2.14 from https://pypi.python.org/packages/74/0e/111a4349e81e2a9846129e0357e154b496559799ec34a6b27bc677247bfa/pycparser-2.14-py2.py3-none-any.whl#md5=130e8dc5b640d9339ee4056da0cdc73a (from -r test.txt (line 1)):
        Expected sha256 7959b4a74abdc27b312fed1c21e6caf9309ce0b29ea86b591fd2e99ecdf27f73
             Got        52bcedd9180999fc7f3128b4b89ce638ffc0ffcbd136873379d5a37e4f9e7932

It's also worth noting that this problem isn't limited to pycparser. As more and more packages start using wheels, any package that introduces a wheel to an existing package version on an index will break clients who have pinned the hash of the .tar.gz source distribution.

I know there are reasons to prefer a wheel over a .tar.gz source distribution. However, if I've pinned the hash of a package and a file with that hash is still available on the package index, as an end-user I'd expect pip to try harder to use that file instead of aborting with a hash mismatch due to the introduction of a (more appropriate) wheel. This feels more user friendly because it won't fail as often. Is this a reasonable request?

[xavfernandez]

This looks like a duplicate of #3634

[indygreg]

I agree this is a dupe of #3634.