Resolver preformance regression in 21.2

[bblommers]

Description

As of 21.2, the resolver decides it needs to download every possible version, to determine which version is compatible with other requirements., for multiple dependencies.

Note that this works fine for some dependencies (coverage, pytest_cov), as it only downloads a few.
For dependencies botocore and boto3, however, it downloads every possible version.

A minimal reproducible example can be found here:
https://github.com/bblommers/pipresolverbug

Not sure what the offending dependency/installation method is - I could only reproduce it for this specific workflow.

The CI for this project runs this specific example for both 21.1 and 21.2.
Version 21.1 completes in 15 seconds - 21.2 is still running after 20 minutes.

The CI logs can be found here:
https://github.com/bblommers/pipresolverbug/actions/runs/1064350454

Expected behavior

To only download the latest version of a dependency

pip version

21.2

Python version

3.7

OS

Linux (Ubuntu)

How to Reproduce

git clone git@github.com:bblommers/pipresolverbug.git
pip install -r req.txt

Output

Collecting botocore>=1.12.201
  Using cached botocore-1.21.2-py3-none-any.whl (7.7 MB)
Collecting boto3>=1.9.201
  Using cached boto3-1.18.1-py3-none-any.whl (131 kB)
Collecting botocore>=1.12.201
  Using cached botocore-1.21.1-py3-none-any.whl (7.7 MB)
Collecting boto3>=1.9.201
  Using cached boto3-1.18.0-py3-none-any.whl (131 kB)
Collecting botocore>=1.12.201
  Using cached botocore-1.21.0-py3-none-any.whl (7.7 MB)
Collecting boto3>=1.9.201
  Using cached boto3-1.17.112-py2.py3-none-any.whl (131 kB)
Collecting s3transfer<0.5.0,>=0.4.0
  Using cached s3transfer-0.4.2-py2.py3-none-any.whl (79 kB)
Collecting botocore>=1.12.201
  Using cached botocore-1.20.112-py2.py3-none-any.whl (7.7 MB)
INFO: pip is looking at multiple versions of boto3 to determine which version is compatible with other requirements. This could take a while.
Collecting boto3>=1.9.201
  Using cached boto3-1.17.111-py2.py3-none-any.whl (131 kB)
Collecting botocore>=1.12.201

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

This is not a bug. Please kindly read this section of the documentation: https://pip.pypa.io/en/stable/user_guide/#dependency-resolution-backtracking

To propose improvements to the dependency resolution algorithm, please see #9187.

[uranusjr]

Reopening since I missed the part you said 21.1 worked reasonably well. I'll investigate whether we can bring back the 21.1 performance for you.