error "unknown url type" with pip >= 23.2

[moritz2k]

For some packages I get an error when checking them since PIP version 23.2
Example Package: Levenshtein
Also affected: Pillow, PyQt5-sip, cchardet, certifi, charset-normalizer, cryptography, deepdiff, exceptiongroup, filelock, lxml, numpy, pandas,...
Working packages: Brotli, ImageHash, PyQt5, PySocks, beautifulsoup4,...

Environment:

• Python 3.11.4
• virtaulenv
• Windows
• Installed packages:

Package    Version
---------- -------
anytree    2.9.0
cachetools 5.3.1
colorama   0.4.6
johnnydep  1.20.2
oyaml      1.0
packaging  23.1
pip        23.2.1
PyYAML     6.0.1
setuptools 68.0.0
six        1.16.0
structlog  23.1.0
tabulate   0.9.0
toml       0.10.2
wheel      0.41.0
wimpy      0.6

Command:
py -m johnnydep --verbose 1 Levenshtein --no-deps --output-format json

Output:

2023-08-17 11:25:56 [info     ] init johnnydist                [johnnydep.lib] dist=Levenshtein parent=None
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\repos\py-package-manager\.venv\Lib\site-packages\johnnydep\__main__.py", line 4, in <module>
    main()
  File "C:\repos\py-package-manager\.venv\Lib\site-packages\johnnydep\cli.py", line 128, in main
    dist = JohnnyDist(
           ^^^^^^^^^^^
  File "C:\repos\py-package-manager\.venv\Lib\site-packages\johnnydep\lib.py", line 83, in __init__
    self.import_names, self.metadata, self.entry_points = _get_info(
                                                          ^^^^^^^^^^
  File "C:\repos\py-package-manager\.venv\Lib\site-packages\cachetools\__init__.py", line 702, in wrapper
    v = func(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\py-package-manager\.venv\Lib\site-packages\johnnydep\lib.py", line 519, in _get_info
    data = pipper.get(
           ^^^^^^^^^^^
  File "C:\repos\py-package-manager\.venv\Lib\site-packages\cachetools\__init__.py", line 737, in wrapper
    v = func(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\py-package-manager\.venv\Lib\site-packages\johnnydep\pipper.py", line 236, in get
    target, _headers = _download_dist(url, scratch_file, index_url, extra_index_url)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\py-package-manager\.venv\Lib\site-packages\johnnydep\pipper.py", line 105, in _download_dist
    target, _headers = urlretrieve(url, scratch_file, auth=auth)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\py-package-manager\.venv\Lib\site-packages\johnnydep\compat.py", line 38, in urlretrieve
    res = opener.open(url, data=data)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\urllib\request.py", line 503, in open
    req = Request(fullurl, data)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\urllib\request.py", line 322, in __init__
    self.full_url = url
    ^^^^^^^^^^^^^
  File "C:\Python311\Lib\urllib\request.py", line 348, in full_url
    self._parse()
  File "C:\Python311\Lib\urllib\request.py", line 377, in _parse
    raise ValueError("unknown url type: %r" % self.full_url)
ValueError: unknown url type: 'Levenshtein-0.21.1-cp311-cp311-win_amd64.whl.metadata'

Expected Output (PIP 23.1.2):

2023-08-17 11:26:36 [info     ] init johnnydist                [johnnydep.lib] dist=Levenshtein parent=None
[
  {
    "name": "levenshtein",
    "summary": "Python extension for computing string edit distances and similarities."
  }
]

[wimglenn]

Thanks, there was some regression due to pip starting to use https://peps.python.org/pep-0658/
Quick fix for now released in johnnydep v1.20.3. The next release will take advantage of a new pip feature to completely rework how johnnydep interacts with the pip subprocess, so that these regressions are less likely.

[moritz2k]

Thanks @wimglenn for the quick reaction, works now!