wheel on pypi.org not found when using --extra-index-url that includes non-compatible wheels

[altendky]

I use a --extra-index-url that contains mostly arm and risc wheels not avilable from pypi.org, or compilations for new versions of python etc. When a version is found at the extra index without it providing a usable wheel, the applicable wheel from pypi.org is not used.

$ python -m venv venv

$ venv/bin/python --version --version
Python 3.11.6 (main, Oct 13 2023, 12:34:41) [GCC 11.4.0]

Installing the latest main uv as of now.

$ venv/bin/pip install 'uv @ git+https://github.com/astral-sh/uv@ba14f6967639a75e3c0781659f646bd95beafdab'
Collecting uv@ git+https://github.com/astral-sh/uv@ba14f6967639a75e3c0781659f646bd95beafdab
  Using cached uv-0.1.22-py3-none-linux_x86_64.whl
Installing collected packages: uv
Successfully installed uv-0.1.22

[notice] A new release of pip is available: 23.2.1 -> 24.0
[notice] To update, run: /home/altendky/tmp/p/venv/bin/python -m pip install --upgrade pip

Listing the v1.1.4 wheel that should get used from pypi.org.

$ curl --silent https://pypi.org/simple/chiavdf/ | grep -e '1\.1\.4' | grep -e cp310 |grep -e manylinux | grep -e x86_64
<a href="https://files.pythonhosted.org/packages/6f/84/0d86567374a45e8172021c1eb751e9a00e9ea4d563af4be969cc4da46566/chiavdf-1.1.4-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl#sha256=986673742982baa3afb7ff0e9580e23b7b7c60e367eac8c63ffbceb07bf702b1" data-requires-python="&gt;=3.8" data-dist-info-metadata="sha256=c1aa82178a5c9f08469dbe3ff54829b9682414838466b37950a139e744d615c7" data-core-metadata="sha256=c1aa82178a5c9f08469dbe3ff54829b9682414838466b37950a139e744d615c7">chiavdf-1.1.4-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl</a><br />

Listing the wrong v1.1.4 wheels on pypi.chia.net that seem to cause trouble.

$ curl --silent https://pypi.chia.net/simple/chiavdf/ | grep '1\.1\.4'
<a href="https://download.chia.net/simple/chiavdf/chiavdf-1.1.4-cp310-cp310-linux_riscv64.whl">chiavdf-1.1.4-cp310-cp310-linux_riscv64.whl</a>                                2024-03-08T18:07:08.000Z      90 KB
<a href="https://download.chia.net/simple/chiavdf/chiavdf-1.1.4-cp311-cp311-linux_riscv64.whl">chiavdf-1.1.4-cp311-cp311-linux_riscv64.whl</a>                                2024-03-08T18:07:09.000Z      91 KB
<a href="https://download.chia.net/simple/chiavdf/chiavdf-1.1.4-cp38-cp38-linux_riscv64.whl">chiavdf-1.1.4-cp38-cp38-linux_riscv64.whl</a>                                  2024-03-08T18:07:10.000Z      90 KB
<a href="https://download.chia.net/simple/chiavdf/chiavdf-1.1.4-cp39-cp39-linux_riscv64.whl">chiavdf-1.1.4-cp39-cp39-linux_riscv64.whl</a>                                  2024-03-08T18:07:11.000Z      91 KB

Failing to install the package using uv and --extra-index-url.

$ venv/bin/python -m uv pip install chiavdf==1.1.4 --extra-index-url https://pypi.chia.net/simple/
  × No solution found when resolving dependencies:
  ╰─▶ Because chiavdf==1.1.4 is unusable because no wheels are available with a matching Python ABI and you require chiavdf==1.1.4, we can conclude that the requirements are unsatisfiable.

Successfully installing the package using uv without --extra-index-url.

$ venv/bin/python -m uv pip install chiavdf==1.1.4 
Resolved 1 package in 118ms
Installed 1 package in 9ms
 + chiavdf==1.1.4

Pip working in the problematic case, albeit with a --force-reinstall added.

$ venv/bin/python -m pip install chiavdf==1.1.4 --force-reinstall --extra-index-url https://pypi.chia.net/simple/
Looking in indexes: https://pypi.org/simple, https://pypi.chia.net/simple/
Collecting chiavdf==1.1.4
  Obtaining dependency information for chiavdf==1.1.4 from https://files.pythonhosted.org/packages/f2/b0/5e103b9d9d0189c0eaa3061661740ae727f5ea75a67cc6c8bb74d235497e/chiavdf-1.1.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading chiavdf-1.1.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (20 kB)
Downloading chiavdf-1.1.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (477 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 477.3/477.3 kB 3.6 MB/s eta 0:00:00
Installing collected packages: chiavdf
  Attempting uninstall: chiavdf
    Found existing installation: chiavdf 1.1.4
    Uninstalling chiavdf-1.1.4:
      Successfully uninstalled chiavdf-1.1.4
Successfully installed chiavdf-1.1.4

[notice] A new release of pip is available: 23.2.1 -> 24.0
[notice] To update, run: /home/altendky/tmp/p/venv/bin/python -m pip install --upgrade pip

[charliermarsh]

Thanks for the clear write-up. This is described here: https://github.com/astral-sh/uv/blob/main/PIP_COMPATIBILITY.md#packages-that-exist-on-multiple-indexes.

There's also discussion on it in a few existing issues:

• Add support for pinning a package to a specific index #171 (comment)
• uv fails to use extra index url #1377
• --extra-index-url failing when package not in --index-url (polygraphy 0.47.1) #1600
• uv pip install for package works from --index-url but not from --extra-index-url #1451
• tweak the order of index priority #2083

[charliermarsh]

This is currently unsupported, but you can follow along #171.