Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

uv pip compile: markupsafe==3.0.2 has no wheels with a matching Python implementation tag #8922

Closed
daip-yxtay opened this issue Nov 8, 2024 · 4 comments · Fixed by #8961
Closed

uv pip compile: markupsafe==3.0.2 has no wheels with a matching Python implementation tag #8922

daip-yxtay opened this issue Nov 8, 2024 · 4 comments · Fixed by #8961
Assignees
@charliermarsh
Labels
bug Something isn't working

Comments

@daip-yxtay
Copy link

daip-yxtay commented Nov 8, 2024 •
edited
Loading

I'm facing an issue when including pytorch cpu index url. This is despite me using the unsafe-best-match index strategy. However, it is successful if I also specify the default pypi index url first. I suspect this is a bug.

#7553

I have seen this other issue mentioned similar error with another package. However, the suggestion to use --python-platform option doesn't work regardless what value I used.

Below is the miminal code.

# constraints.txt
MarkupSafe~=3.0.2
Jinja2~=3.1.4
torch~=2.4.0

# requirements.in
torch

It works if pytorch cpu index url is not used.

uv pip compile \
  --no-annotate \
  --index-strategy unsafe-best-match \
  --constraint constraints.txt \
  requirements.in

Resolved 22 packages in 42ms
# This file was autogenerated by uv via the following command:
#    uv pip compile --no-annotate --index-strategy unsafe-best-match --constraint constraints.txt requirements.in
filelock==3.16.1
fsspec==2024.10.0
jinja2==3.1.4
markupsafe==3.0.2
mpmath==1.3.0
networkx==3.4.2
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==9.1.0.70
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-nccl-cu12==2.20.5
nvidia-nvjitlink-cu12==12.6.77
nvidia-nvtx-cu12==12.1.105
sympy==1.13.3
torch==2.4.1
triton==3.0.0
typing-extensions==4.12.2

But fails if pytorch cpu index url is used.

uv pip compile \
  --no-annotate \
  --index-strategy unsafe-best-match \
  --extra-index-url https://download.pytorch.org/whl/cpu \
  --constraint constraints.txt \
  requirements.in

  × No solution found when resolving dependencies:
  ╰─▶ Because only the following versions of torch are available:
          torch<=2.4.0
          torch==2.4.0+cpu
          torch==2.4.1
          torch==2.4.1+cpu
          torch>2.5.dev0
      and torch==2.4.0 has no wheels with a matching Python implementation
      tag, we can conclude that torch>=2.4.0,<2.4.0+cpu cannot be used. (1)

      Because markupsafe==3.0.2 has no wheels with a matching Python
      implementation tag and only markupsafe<=3.0.2 is available, we can
      conclude that markupsafe>=3.0.2 cannot be used.
      And because jinja2==3.1.4 depends on markupsafe>=3.0.2, we can conclude
      that jinja2==3.1.4 cannot be used.
      And because only jinja2<=3.1.4 is available and all of:
          torch==2.4.0+cpu
          torch==2.4.1+cpu
      depend on jinja2>=3.1.4, we can conclude that all of:
          torch==2.4.0+cpu
          torch==2.4.1+cpu
       cannot be used.
      And because we know from (1) that torch>=2.4.0,<2.4.0+cpu cannot be
      used, we can conclude that all of:
          torch>=2.4.0,<2.4.1
          torch>2.4.1,<2.5.dev0
       cannot be used.
      And because torch==2.4.1 has no wheels with a matching Python
      implementation tag and you require torch>=2.4.0,<2.5.dev0, we can
      conclude that your requirements are unsatisfiable.

      hint: torch was requested with a pre-release marker (e.g., all of:
          torch>2.4.0,<2.4.0+cpu
          torch>2.4.0+cpu,<2.4.1
          torch>2.4.1,<2.4.1+cpu
          torch>2.4.1+cpu,<2.5.dev0
      ), but pre-releases weren't enabled (try: `--prerelease=allow`)

      hint: markupsafe was requested with a pre-release marker (e.g.,
      markupsafe>3.0.2,<3.1.dev0), but pre-releases weren't enabled (try:
      `--prerelease=allow`)

      hint: jinja2 was requested with a pre-release marker (e.g.,
      jinja2>3.1.4,<3.2.dev0), but pre-releases weren't enabled (try:
      `--prerelease=allow`)

It works if I add the default pypi index url first.

uv pip compile \
  --no-annotate \
  --index-strategy unsafe-best-match \
  --extra-index-url https://pypi.org/simple \
  --extra-index-url https://download.pytorch.org/whl/cpu \
  --constraint constraints.txt \
  requirements.in

Resolved 9 packages in 2.04s
# This file was autogenerated by uv via the following command:
#    uv pip compile --no-annotate --index-strategy unsafe-best-match --constraint constraints.txt requirements.in
filelock==3.16.1
fsspec==2024.10.0
jinja2==3.1.4
markupsafe==3.0.2
mpmath==1.3.0
networkx==3.4.2
sympy==1.13.3
torch==2.4.1+cpu
typing-extensions==4.12.2

However, it gives the same error if the default pypi index url is specified last.

uv pip compile \
  --no-annotate \
  --index-strategy unsafe-best-match \
  --extra-index-url https://download.pytorch.org/whl/cpu \
  --extra-index-url https://pypi.org/simple \
  --constraint constraints.txt \
  requirements.in

pip-tools has no issue.

pip-compile \
  --no-annotate \
  --extra-index-url https://download.pytorch.org/whl/cpu \
  --constraint constraints.txt \
  requirements.in

WARNING: --strip-extras is becoming the default in version 8.0.0. To silence this warning, either use --strip-extras to opt into the new default or use --no-strip-extras to retain the existing behavior.
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --constraint=constraints.txt --extra-index-url=https://download.pytorch.org/whl/cpu --no-annotate requiremenst.in
#
--extra-index-url https://download.pytorch.org/whl/cpu

filelock==3.16.1
fsspec==2024.10.0
jinja2==3.1.4
markupsafe==3.0.2
mpmath==1.3.0
networkx==3.4.2
sympy==1.13.3
torch==2.4.1+cpu
typing-extensions==4.12.2

If I were to downgrade markupsafe==3.0.1, it works.

# constraints.txt
MarkupSafe~=3.0.1
Jinja2~=3.1.4
torch~=2.4.0

uv pip compile \
  --no-annotate \
  --index-strategy unsafe-best-match \
  --extra-index-url https://download.pytorch.org/whl/cpu \
  --constraint constraints.txt \
  requirements.in

Resolved 9 packages in 3.81s
# This file was autogenerated by uv via the following command:
#    uv pip compile --no-annotate --index-strategy unsafe-best-match --constraint constraints.txt requirements.in
filelock==3.16.1
fsspec==2024.10.0
jinja2==3.1.4
markupsafe==3.0.1
mpmath==1.3.0
networkx==3.4.2
sympy==1.13.3
torch==2.4.1+cpu
typing-extensions==4.12.2

versions

python==3.10.12
pip-tools==7.4.1
uv==0.4.30

verbose output

DEBUG uv 0.4.30
DEBUG Starting Python discovery for a default Python
DEBUG Looking for exact match for request a default Python
DEBUG Searching for default Python interpreter in managed installations or system path
DEBUG Found `cpython-3.10.12-linux-x86_64-gnu` at `/local_disk0/.ephemeral_nfs/envs/pythonEnv-361198c6-c8d6-4e0d-a774-f0b153253d6b/bin/python3` (active virtual environment)
DEBUG Using Python 3.10.12 interpreter at /local_disk0/.ephemeral_nfs/envs/pythonEnv-361198c6-c8d6-4e0d-a774-f0b153253d6b/bin/python3 for builds
DEBUG Using request timeout of 30s
DEBUG Solving with installed Python version: 3.10.12
DEBUG Solving with target Python version: >=3.10.12
DEBUG Adding direct dependency: torch*
DEBUG Adding direct dependency: torch>=2.4.0, <2.5.dev0
DEBUG No cache entry for: https://download.pytorch.org/whl/cpu/torch/
DEBUG Found fresh response for: https://pypi.org/simple/torch/
DEBUG Found fresh response for: https://pypi.org/simple/torch/
DEBUG Searching for a compatible version of torch (>=2.4.0, <2.5.dev0)
DEBUG Selecting: torch==2.4.1+cpu [compatible] (torch-2.4.1+cpu-cp310-cp310-linux_x86_64.whl)
DEBUG Found stale response for: https://download.pytorch.org/whl/cpu/torch-2.4.1%2Bcpu-cp310-cp310-linux_x86_64.whl#sha256=833490a28ac156762ed6adaa7c695879564fa2fd0dc51bcf3fdb2c7b47dc55e6
DEBUG Sending revalidation request for: https://download.pytorch.org/whl/cpu/torch-2.4.1%2Bcpu-cp310-cp310-linux_x86_64.whl#sha256=833490a28ac156762ed6adaa7c695879564fa2fd0dc51bcf3fdb2c7b47dc55e6
DEBUG Found not-modified response for: https://download.pytorch.org/whl/cpu/torch-2.4.1%2Bcpu-cp310-cp310-linux_x86_64.whl#sha256=833490a28ac156762ed6adaa7c695879564fa2fd0dc51bcf3fdb2c7b47dc55e6
DEBUG Adding transitive dependency for torch==2.4.1+cpu: filelock*
DEBUG Adding transitive dependency for torch==2.4.1+cpu: typing-extensions>=4.8.0
DEBUG Adding transitive dependency for torch==2.4.1+cpu: sympy*
DEBUG Adding transitive dependency for torch==2.4.1+cpu: networkx*
DEBUG Adding transitive dependency for torch==2.4.1+cpu: jinja2*
DEBUG Adding transitive dependency for torch==2.4.1+cpu: jinja2>=3.1.4, <3.2.dev0
DEBUG Adding transitive dependency for torch==2.4.1+cpu: fsspec*
DEBUG No cache entry for: https://download.pytorch.org/whl/cpu/filelock/
DEBUG No cache entry for: https://download.pytorch.org/whl/cpu/typing-extensions/
DEBUG No cache entry for: https://download.pytorch.org/whl/cpu/sympy/
DEBUG No cache entry for: https://download.pytorch.org/whl/cpu/networkx/
DEBUG No cache entry for: https://download.pytorch.org/whl/cpu/jinja2/
DEBUG No cache entry for: https://download.pytorch.org/whl/cpu/fsspec/
DEBUG Found fresh response for: https://pypi.org/simple/networkx/
DEBUG Found fresh response for: https://pypi.org/simple/networkx/
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/b9/54/dd730b32ea14ea797530a4479b2ed46a6fb250f682a9cfb997e968bf0261/networkx-3.4.2-py3-none-any.whl.metadata
DEBUG Found fresh response for: https://pypi.org/simple/typing-extensions/
DEBUG Found fresh response for: https://pypi.org/simple/typing-extensions/
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/26/9f/ad63fc0248c5379346306f8668cda6e2e2e9c95e01216d2b8ffd9ff037d0/typing_extensions-4.12.2-py3-none-any.whl.metadata
DEBUG Found fresh response for: https://pypi.org/simple/fsspec/
DEBUG Found fresh response for: https://pypi.org/simple/fsspec/
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/c6/b2/454d6e7f0158951d8a78c2e1eb4f69ae81beb8dca5fee9809c6c99e9d0d0/fsspec-2024.10.0-py3-none-any.whl.metadata
DEBUG Found fresh response for: https://pypi.org/simple/sympy/
DEBUG Found fresh response for: https://pypi.org/simple/sympy/
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/99/ff/c87e0622b1dadea79d2fb0b25ade9ed98954c9033722eb707053d310d4f3/sympy-1.13.3-py3-none-any.whl.metadata
DEBUG Found fresh response for: https://pypi.org/simple/filelock/
DEBUG Found fresh response for: https://pypi.org/simple/filelock/
DEBUG Searching for a compatible version of filelock (*)
DEBUG Selecting: filelock==3.16.1 [compatible] (filelock-3.16.1-py3-none-any.whl)
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/b9/f8/feced7779d755758a52d1f6635d990b8d98dc0a29fa568bbe0625f18fdf3/filelock-3.16.1-py3-none-any.whl.metadata
DEBUG Searching for a compatible version of typing-extensions (>=4.8.0)
DEBUG Selecting: typing-extensions==4.12.2 [compatible] (typing_extensions-4.12.2-py3-none-any.whl)
DEBUG Searching for a compatible version of sympy (*)
DEBUG Selecting: sympy==1.13.3 [compatible] (sympy-1.13.3-py3-none-any.whl)
DEBUG Adding transitive dependency for sympy==1.13.3: mpmath>=1.1.0, <1.4
DEBUG Searching for a compatible version of networkx (*)
DEBUG Selecting: networkx==3.4.2 [compatible] (networkx-3.4.2-py3-none-any.whl)
DEBUG No cache entry for: https://download.pytorch.org/whl/cpu/mpmath/
DEBUG Found fresh response for: https://pypi.org/simple/jinja2/
DEBUG Found fresh response for: https://pypi.org/simple/jinja2/
DEBUG Searching for a compatible version of jinja2 (>=3.1.4, <3.2.dev0)
DEBUG Selecting: jinja2==3.1.4 [compatible] (jinja2-3.1.4-py3-none-any.whl)
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/31/80/3a54838c3fb461f6fec263ebf3a3a41771bd05190238de3486aae8540c36/jinja2-3.1.4-py3-none-any.whl.metadata
DEBUG Adding transitive dependency for jinja2==3.1.4: markupsafe>=2.0
DEBUG Adding transitive dependency for jinja2==3.1.4: markupsafe>=3.0.2, <3.1.dev0
DEBUG Searching for a compatible version of fsspec (*)
DEBUG Selecting: fsspec==2024.10.0 [compatible] (fsspec-2024.10.0-py3-none-any.whl)
DEBUG No cache entry for: https://download.pytorch.org/whl/cpu/markupsafe/
DEBUG Found fresh response for: https://pypi.org/simple/mpmath/
DEBUG Found fresh response for: https://pypi.org/simple/mpmath/
DEBUG Searching for a compatible version of mpmath (>=1.1.0, <1.4)
DEBUG Selecting: mpmath==1.3.0 [compatible] (mpmath-1.3.0-py3-none-any.whl)
DEBUG Found stale response for: https://download.pytorch.org/whl/mpmath-1.3.0-py3-none-any.whl#sha256=a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c
DEBUG Sending revalidation request for: https://download.pytorch.org/whl/mpmath-1.3.0-py3-none-any.whl#sha256=a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c
DEBUG Found not-modified response for: https://download.pytorch.org/whl/mpmath-1.3.0-py3-none-any.whl#sha256=a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c
DEBUG Found fresh response for: https://pypi.org/simple/markupsafe/
DEBUG Found fresh response for: https://pypi.org/simple/markupsafe/
DEBUG Searching for a compatible version of markupsafe (>=3.0.2, <3.1.dev0)
DEBUG Searching for a compatible version of markupsafe (>3.0.2, <3.1.dev0)
DEBUG No compatible version found for: markupsafe
DEBUG Searching for a compatible version of filelock (*)
DEBUG Selecting: filelock==3.16.1 [compatible] (filelock-3.16.1-py3-none-any.whl)
DEBUG Searching for a compatible version of typing-extensions (>=4.8.0)
DEBUG Selecting: typing-extensions==4.12.2 [compatible] (typing_extensions-4.12.2-py3-none-any.whl)
DEBUG Searching for a compatible version of sympy (*)
DEBUG Selecting: sympy==1.13.3 [compatible] (sympy-1.13.3-py3-none-any.whl)
DEBUG Searching for a compatible version of networkx (*)
DEBUG Selecting: networkx==3.4.2 [compatible] (networkx-3.4.2-py3-none-any.whl)
DEBUG Searching for a compatible version of jinja2 (>3.1.4, <3.2.dev0)
DEBUG No compatible version found for: jinja2
DEBUG Searching for a compatible version of torch (>=2.4.0, <2.4.1+cpu | >2.4.1+cpu, <2.5.dev0)
DEBUG Searching for a compatible version of torch (>=2.4.0, <2.4.1 | >2.4.1, <2.4.1+cpu | >2.4.1+cpu, <2.5.dev0)
DEBUG Selecting: torch==2.4.0+cpu [compatible] (torch-2.4.0+cpu-cp310-cp310-linux_x86_64.whl)
DEBUG Found stale response for: https://download.pytorch.org/whl/cpu/torch-2.4.0%2Bcpu-cp310-cp310-linux_x86_64.whl#sha256=0e59377b27823dda6d26528febb7ca06fc5b77816eaa58b4420cc8785e33d4ce
DEBUG Sending revalidation request for: https://download.pytorch.org/whl/cpu/torch-2.4.0%2Bcpu-cp310-cp310-linux_x86_64.whl#sha256=0e59377b27823dda6d26528febb7ca06fc5b77816eaa58b4420cc8785e33d4ce
DEBUG Found not-modified response for: https://download.pytorch.org/whl/cpu/torch-2.4.0%2Bcpu-cp310-cp310-linux_x86_64.whl#sha256=0e59377b27823dda6d26528febb7ca06fc5b77816eaa58b4420cc8785e33d4ce
DEBUG Adding transitive dependency for torch==2.4.0+cpu: filelock*
DEBUG Adding transitive dependency for torch==2.4.0+cpu: typing-extensions>=4.8.0
DEBUG Adding transitive dependency for torch==2.4.0+cpu: sympy*
DEBUG Adding transitive dependency for torch==2.4.0+cpu: networkx*
DEBUG Adding transitive dependency for torch==2.4.0+cpu: jinja2*
DEBUG Adding transitive dependency for torch==2.4.0+cpu: jinja2>=3.1.4, <3.2.dev0
DEBUG Adding transitive dependency for torch==2.4.0+cpu: fsspec*
DEBUG Searching for a compatible version of torch (>=2.4.0, <2.4.0+cpu | >2.4.0+cpu, <2.4.1 | >2.4.1, <2.4.1+cpu | >2.4.1+cpu, <2.5.dev0)
DEBUG Searching for a compatible version of torch (>2.4.0, <2.4.0+cpu | >2.4.0+cpu, <2.4.1 | >2.4.1, <2.4.1+cpu | >2.4.1+cpu, <2.5.dev0)
DEBUG No compatible version found for: torch
  × No solution found when resolving dependencies:
  ╰─▶ Because only the following versions of torch are available:
          torch<=2.4.0
          torch==2.4.0+cpu
          torch==2.4.1
          torch==2.4.1+cpu
          torch>2.5.dev0
      and torch==2.4.0 has no wheels with a matching Python implementation
      tag, we can conclude that torch>=2.4.0,<2.4.0+cpu cannot be used. (1)

      Because markupsafe==3.0.2 has no wheels with a matching Python
      implementation tag and only markupsafe<=3.0.2 is available, we can
      conclude that markupsafe>=3.0.2 cannot be used.
      And because jinja2==3.1.4 depends on markupsafe>=3.0.2, we can conclude
      that jinja2==3.1.4 cannot be used.
      And because only jinja2<=3.1.4 is available and all of:
          torch==2.4.0+cpu
          torch==2.4.1+cpu
      depend on jinja2>=3.1.4, we can conclude that all of:
          torch==2.4.0+cpu
          torch==2.4.1+cpu
       cannot be used.
      And because we know from (1) that torch>=2.4.0,<2.4.0+cpu cannot be
      used, we can conclude that all of:
          torch>=2.4.0,<2.4.1
          torch>2.4.1,<2.5.dev0
       cannot be used.
      And because torch==2.4.1 has no wheels with a matching Python
      implementation tag and you require torch>=2.4.0,<2.5.dev0, we can
      conclude that your requirements are unsatisfiable.

      hint: torch was requested with a pre-release marker (e.g., all of:
          torch>2.4.0,<2.4.0+cpu
          torch>2.4.0+cpu,<2.4.1
          torch>2.4.1,<2.4.1+cpu
          torch>2.4.1+cpu,<2.5.dev0
      ), but pre-releases weren't enabled (try: `--prerelease=allow`)

      hint: markupsafe was requested with a pre-release marker (e.g.,
      markupsafe>3.0.2,<3.1.dev0), but pre-releases weren't enabled (try:
      `--prerelease=allow`)

      hint: jinja2 was requested with a pre-release marker (e.g.,
      jinja2>3.1.4,<3.2.dev0), but pre-releases weren't enabled (try:
      `--prerelease=allow`)
@charliermarsh
Copy link
Member

My guess is that this is because 3.0.2 exists on both indexes, but on the Torch index, it only has a Python 3.13 manylinux wheel. Perhaps we aren't looking at PyPI once we find that it has the same version, but at lower priority?

@charliermarsh charliermarsh added the bug Something isn't working label Nov 8, 2024
@charliermarsh
Copy link
Member

(It does look like a bug though, in that case.)

@charliermarsh charliermarsh self-assigned this Nov 8, 2024
@charliermarsh charliermarsh mentioned this issue Nov 9, 2024
Merged
charliermarsh added a commit that referenced this issue Nov 9, 2024
@charliermarsh
Prefer compatible to incompatible distributions when packages exist o…
363c589 
…n multiple indexes (#8961)

## Summary

At time of writing, `markupsafe==3.0.2` exists on the PyTorch index, but
there's
only a single wheel:


`MarkupSafe-3.0.2-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl`

Meanwhile, there are a large number of wheels on PyPI for the same
version. If the
user is on Python 3.12, and we return the incompatible PyTorch wheel
without
considering the PyPI wheels, PubGrub will mark 3.0.2 as an incompatible
version,
even though there are compatible wheels on PyPI.

Closes #8922.
@charliermarsh
Copy link
Member

Fixed in the next release.

konstin pushed a commit that referenced this issue Nov 10, 2024
@charliermarsh @konstin
Prefer compatible to incompatible distributions when packages exist o…
d4cfb0b 
…n multiple indexes (#8961)

## Summary

At time of writing, `markupsafe==3.0.2` exists on the PyTorch index, but
there's
only a single wheel:


`MarkupSafe-3.0.2-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl`

Meanwhile, there are a large number of wheels on PyPI for the same
version. If the
user is on Python 3.12, and we return the incompatible PyTorch wheel
without
considering the PyPI wheels, PubGrub will mark 3.0.2 as an incompatible
version,
even though there are compatible wheels on PyPI.

Closes #8922.
@yxtay
Copy link

yxtay commented Nov 11, 2024

Wow, thanks for the extremely quick response! 👍

@robinjhuang robinjhuang mentioned this issue Nov 20, 2024
Merged
@jfaust jfaust mentioned this issue Dec 4, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@charliermarsh charliermarsh

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Prefer compatible to incompatible distributions when packages exist on multiple indexes astral-sh/uv
3 participants
@charliermarsh @yxtay @daip-yxtay