Pull in Pip fixes.

[jsirois]

Some Pex users are encountering resolve issues fixed in newer versions of Pip. Pex has been stuck on the last version of Pip to support Python 2.7 (20.3.4) in order to continue to support Python 2.7 customers. Either Pex has to cut those customers loose or it has to support both old and new Pips.

The basic scenarios:

1. Cut loose - fend for yourself: Pex simply bumps its major version and moves on; Python 2.7 users get no bug fixes or features and are stuck on the last version of Pex to vendor Pip 20.3.4.
2. Bifurcate with support: Like 1, except Pex cuts a long-term branch where fixes and releases for the Pex 2.1.x / Pip 20.3.4 series go on in parallel to Pex 3.x / Pip ~latest development and releases off of main.
3. Support two versions of Pip in the same Pex release: This could keep the 20.3.4 Pip vendoring in the released Pex but just-in-time resolve a newer version of Pip using the older vendored Pip. The use of the newer Pip could either be controlled via flag or via auto-detect from interpreter constraints / interpreters used / platforms.

Obviously, 1 is easiest from a maintainer perspective, but it violates an implicit contract with users. It would be better to phase out Python 2.7 / 3.5 / 3.6 / etc. support in a controlled way independent of needing to upgrade Pip to get Pip bug-fixes. As the current ~lone maintainer, I'm partial to 3 to both meet users needs and have a compact codebase to maintain. I fear 2 would lead to more complicated maintenance and missed / partial fixes.

[jsirois]

The motivating slow lock resolves were reported variously through #1798 on behalf of Pants users.

[jsirois]

This upgrade looks like it will close #1526, #1527 and #1528.

[jsirois]

The only apparent differences found in Pip 22.2.2 are:

1. An extra Wheel model method that needs to be runtime patched for --style universal locks. This patch is a noop against 20.3.4-patched; so this works out well.
2. Pip now attempts to use the json API defined in https://peps.python.org/pep-0691/. This API is implemented by PyPI; so Pip logs from PyPI downloads no longer contain hashes, they are hidden in a fetch of https://pypi.org/simple/<project name>/ with Accept: application/vnd.pypi.simple.v1+json.

1 is patched and working, and 2 just leads to less efficient locking. The fall-back download-and-hash code is now used always. So 2 needs an optimization fix to 1st try to use PEP-691 json API for all locked urls without a hash before falling back to the much more expensive, download all those URLS to hash their contents.

[jsirois]

Ok, wiring up the PEP 691 JSON API was straight forward. This will fail for custom indexes with Bearer auth, but then we just fall back to downloading and hashing. That's slower, but still correct.

This leaves 1 unit and 3 IT failing using the 22.2.2 resolver in existing tests. The unit failure is in bdist_pex and still mysterious and the 3 ITs appear to center around local project / VCS hashing and those are still otherwise mysterious as well.

[jsirois]

Alright, sorted. Supporting Pip 20.3.4-patched + vendored setuptools and wheel by default and Pip 22.2.2 + setuptools 65.0.0 + wheel 0.37.1 (all latest as of today), when --pip-version is requested turns out to be viable. There is 1 added line of runtime patching code to Pip and a new utility to get hashes via PEP-691 that is only engaged for new log lines emitted by new Pip. It should be the case that Pex can just add to the PipVersion enum every now and again to expand to newer Pips while retaining support for 20.3.4-patched ~indefinitely at the User's prerogative. Pex3 can just flip the default from vendored to latest supported Pip, but can continue to support users until Redhat 2.7 support goes away or some similar proxy for truly EOL.

[jsirois]

Noting I hit this bug using 22.2.2 in ITs sporadically: pypa/pip#11340 Users will too and that's fine. The answer will be to add a new --pip-version once they release the fix. It does beg the question though of how many Pip versions Pex will end up having in the choices for --pip-version. That set could grow pretty big - 100s. Maybe that's fine. Alternatively Pex could get fancy and allow for implicit latest in that set when version components are left off. --pip-version 22 would snap to 22.2.3 when Pip releases that and Pex then adds / verifies support for 22.2.3.