pip CLI emits DeprecationWarning due to use of LegacyVersion

[kitchoi]

Environment

• pip version: 20.3.1
• Python version: 3.9
• OS: MacOSX

Description
pip CLI consistently causes a DeprecationWarning to be emitted from its own vendored packaging. This is not observed in pip 20.2.3

Expected behavior
Expect no DeprecationWarning

How to Reproduce

1. Install a fresh Python environment, upgrade pip to 20.3.1
2. Set environment variable PYTHONWARNINGS to default
3. DeprecationWarnings occur consistently in a pip CLI command.

Output

To create an environment with an older version of pip:

$ python3.9 -m venv test-pip
$ source test-pip/bin/activate

With PYTHONWARNINGS environment variable set to "default", no DeprecationWarnings:

(test-pip) $ echo $PYTHONWARNINGS
default
(test-pip) $ pip list
Package    Version
---------- -------
pip        20.2.3
setuptools 49.2.1
WARNING: You are using pip version 20.2.3; however, version 20.3.1 is available.
You should consider upgrading via the '/Users/kchoi/test-pip/bin/python3.9 -m pip install --upgrade pip' command.

Then upgrade to 20.3.1:

(test-pip) $ pip install -U pip
Collecting pip
  Using cached pip-20.3.1-py2.py3-none-any.whl (1.5 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.2.3
    Uninstalling pip-20.2.3:
      Successfully uninstalled pip-20.2.3
Successfully installed pip-20.3.1

DeprecationWarning appears.

(test-pip) kchoi-mbp2019:~ kchoi$ pip list
/Users/kchoi/test-pip/lib/python3.9/site-packages/pip/_vendor/packaging/version.py:127: DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release
  warnings.warn(

Package    Version
---------- -------
pip        20.3.1
setuptools 49.2.1
/Users/kchoi/test-pip/lib/python3.9/site-packages/pip/_vendor/packaging/version.py:127: DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release
  warnings.warn(

[domdfcoding]

I can reproduce:

Long traceback (click to expand)

$ python3 -Werror -m pip
Traceback (most recent call last):
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_vendor/packaging/version.py", line 57, in parse
    return Version(version)
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_vendor/packaging/version.py", line 298, in __init__
    raise InvalidVersion("Invalid version: '{0}'".format(version))
pip._vendor.packaging.version.InvalidVersion: Invalid version: 'setuptools'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/__main__.py", line 23, in <module>
    from pip._internal.cli.main import main as _main  # isort:skip # noqa
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 10, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py", line 9, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py", line 7, in <module>
    from pip._internal.cli import cmdoptions
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_internal/cli/cmdoptions.py", line 25, in <module>
    from pip._internal.cli.progress_bars import BAR_TYPES
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_internal/cli/progress_bars.py", line 12, in <module>
    from pip._internal.utils.logging import get_indentation
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_internal/utils/logging.py", line 18, in <module>
    from pip._internal.utils.misc import ensure_dir
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_internal/utils/misc.py", line 21, in <module>
    from pip._vendor import pkg_resources
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3252, in <module>
    def _initialize_master_working_set():
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3235, in _call_aside
    f(*args, **kwargs)
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 574, in _build_master
    ws = cls()
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 567, in __init__
    self.add_entry(entry)
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 623, in add_entry
    for dist in find_distributions(entry, True):
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2061, in find_on_path
    path_item_entries = _by_version_descending(filtered)
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2034, in _by_version_descending
    return sorted(names, key=_by_version, reverse=True)
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2032, in _by_version
    return [packaging.version.parse(part) for part in parts]
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2032, in <listcomp>
    return [packaging.version.parse(part) for part in parts]
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_vendor/packaging/version.py", line 59, in parse
    return LegacyVersion(version)
  File "/tmp/pip_test/venv/lib/python3.8/site-packages/pip/_vendor/packaging/version.py", line 127, in __init__
    warnings.warn(
DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release

All that's installed is:

pip==20.3.1
setuptools==50.3.2
wheel==0.35.1

I can narrow it down to:

pip/src/pip/_vendor/pkg_resources/__init__.py

Line 2032 in 27d8687

return [packaging.version.parse(part) for part in parts]

where the value of parts is ['setuptools', '50.3.2', '.dist-info']. Something's not right there.

[domdfcoding]

The same bug has been reported over at pypa/setuptools#2466 and pypa/packaging#368, and there's a draft fix: pypa/setuptools#2478

[mschmitzer]

I think this has been fixed in pypa/setuptools@47b9bf5 . So updating the vendored pkg_resources to 59.1.1 should fix this issue in pip.

[pradyunsg]

Neat. We can finally get rid of the noise here and make this a proper warning.

[pradyunsg]

FWIW, the plan is for this warning to preceed the removal of handling of non-PEP 440 versions.

[uranusjr]

I think this has been fixed in pypa/setuptools@47b9bf5

Oh finally! (I’m happy but also a bit sad seeing this needing a contribution from Jason.)

[wolfch-elsevier]

I still get this warning with Python-3.9.8 and pip version 21.3.1...

[lukasjuhrich]

I think this has been fixed in pypa/setuptools@47b9bf5 . So updating the vendored pkg_resources to 59.1.1 should fix this issue in pip.

In my pip==22.2.2 installation, the vendored pkg_resources version does not contain this change. $VENV_PATH/lib/python3.10/site-packages/pip/_vendor/pkg_resources/__init__.py line 2032 still reads:

        return [packaging.version.parse(part) for part in parts]

I'm not sure how the vendoring process in pip works. Is there any action required besides installing a current pip version, or do we have to wait until a maintainer updates the dependency in pip?

[uranusjr]

IIRC recent setuptools versions change too significantly for pip to reasonably vendor, so someone will need to manually cherry-pick the change in. Feel free.

(Or just use 3.11+ or later where the metadata backend has been switched to importlib.metadata.)

[mitsuhiko]

I have a virtualenv where I get the same. The _by_version_descending function in pkg_resources is invoked by pip with these values:

['pip-23.0.1.dist-info', 'setuptools-65.5.0.dist-info']

Which fails because of this:

parts = itertools.chain(name.split('-'), [ext])
return [packaging.version.parse(part) for part in parts]

So it tries to parse the version pip which fails which in turn then emits this warning.

[pradyunsg]

Consolidating into #11715.