universal resolution occasionally doesn't work with no-build-package when the package is a transient dependency of a dependency included in both dependency-groups and build-system.requires

[DetachHead]

Summary

this issue seems to be similar to #9396 in that it seems to happen randomly and i can't reliably reproduce it, despite spending hours trying to come up with a minimal repro.

from my understanding of universal resolution, if i have a dependency on a package that does not have a single version that can be used for the entire supported requires-python range, multiple versions can be resolved to account for that. however if i try to use this functionality with no-build-package, it seems to sometimes fail to resolve the package:

[project]
name = "asdfads"
description = "asdf"
authors = [ 
    {name = "detachhead", email = "detachhead@users.noreply.github.com"},
]
dependencies = []
requires-python = ">=3.8"   
readme = "README.md"
license = {text = "MIT"}
version = "1.0.0"
 
[dependency-groups] 
docstubs = [
    "docify>=1.0.0"
]

[build-system]
build-backend = "pdm.backend"
requires = [
    "pdm-backend>=2.3.0",
    "docify>=1.0.0",
]

[tool.uv]
default-groups = ["docstubs"]
# transient dependency of docify. needed because there's no single version with wheels for 3.8-3.13 and it fails to build when there's no wheels
no-build-package = ["libcst"]

here, i expect it to select libcst==1.1.0 for python 3.8 and libcst==1.6.0 for python >=3.9 because while 1.1.0 supports python 3.8-3.13, there's no wheels for 3.13 and 1.6.0 only supports python >=3.9. it sometimes works as i expect, since i can see this in the generated lockfile:

[[package]]
name = "docify"
version = "1.1.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
    { name = "libcst", version = "1.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.9'" },
    { name = "libcst", version = "1.6.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.9'" },
    { name = "tqdm" },
]
sdist = { url = "https://files.pythonhosted.org/packages/5c/43/b6b3ed504910ce076e6b7b6659aba4d26e51da3d3116393232278df22284/docify-1.1.0.tar.gz", hash = "sha256:d9475055a330b4edc31be5854d78704a9c231dcf29b4c6e79421e4ce2adb90aa", size = 8610 }
wheels = [
    { url = "https://files.pythonhosted.org/packages/d1/b4/a5ea211025bf8544a1366e04c57efca6c9d1ba45028e53269ce1182bb49d/docify-1.1.0-py3-none-any.whl", hash = "sha256:9b45a82f0da74e97f80d5ace255b417ec289cb2d22d321bfe3124f38a42df68b", size = 8652 },
]

but sometimes it fails to do this

steps to reproduce

1. uv sync --upgrade

output:

Using CPython 3.13.0 interpreter at: C:\Program Files\Python313\python.exe
Removed virtual environment at: .venv
Creating virtual environment at: .venv
  × No solution found when resolving dependencies for split (python_full_version >= '3.8.1' and python_full_version < '3.9'):
      And because libcst==1.4.0 has no usable wheels, we can conclude that libcst>=1.4.0 cannot be used. (1)

      Because only the following versions of libcst are available:
          libcst<=1.1.0
          libcst==1.2.0
          libcst==1.3.0
          libcst==1.3.1
          libcst==1.4.0
          libcst==1.5.0
          libcst==1.5.1
      and libcst>=1.1.0,<=1.3.1 has no usable wheels, we can conclude that libcst>=1.1.0,<=1.3.1 cannot be used.
      And because we know from (1) that libcst>=1.4.0 cannot be used, we can conclude that libcst>=1.1.0 cannot be used.
      And because all versions of docify depend on libcst>=1.1.0, we can conclude that all versions of docify cannot be used.
      And because basedpyright:docstubs depends on docify and your project requires basedpyright:docstubs, we can conclude that your project's requirements are unsatisfiable.

      hint: The `requires-python` value (>=3.8) includes Python versions that are not supported by your dependencies (e.g., libcst>=1.5.0 only supports >=3.9). Consider using a more restrictive

      hint: Wheels are required for `libcst` because building from source is disabled for `libcst` (i.e., with `--no-build-package libcst`)

i realize this probably isn't really a helpful without a consistent repro, but this is the second issue i've run into where uv seems to change its behavior randomly when i haven't changed anything in my project. it's quite frustrating so i would like to try and get to the the bottom of what's causing these issues, so any advice on additional info i can provide / debugging steps would be appreciated. i've tried deleting my .venv, deleting uv.lock and running uv with --no-cache but it doesn't seem to have an impact. my current theory is that it might have something to do with the fact that i upgraded uv from 0.5.20 to 0.5.24, but i can't reliably reproduce it by repeating those steps either :(

Platform

windows 10

Version

0.5.24

Python version

3.13

[charliermarsh]

I can look.

[charliermarsh]

As you guessed it's hard to do much here, I can't reproduce :/

[DetachHead]

i will try playing around with this some more tomorrow. i'll close this for now and will re-open it if i can figure out a way to reliably reproduce it. either way thanks for looking into it

[charliermarsh]

Thanks. If you can figure out a reliable repro I'd love to fix it!

[charliermarsh]

My guess (?) is that it has something to do with --upgrade interacting with the existing lockfile, but it's hard to say.

[BurntSushi]

Apologies for the more narrow comment here, but:

here, i expect it to select libcst==1.1.0 for python 3.8 and libcst==1.6.0 for python >=3.9 because while 1.1.0 supports python 3.8-3.13, there's no wheels for 3.13 and 1.6.0 only supports python >=3.9. it sometimes works as i expect, since i can see this in the generated lockfile:

Have you tried doing something like this?

"libcst==1.1.0 ; python_version == '3.8'",
"libcst>1.1.0 ; python_version >= '3.9'",

Sorry if I'm missing something here. And obviously the non-determinism you're seeing here isn't really addressed by this.

[DetachHead]

yeah that's what i was doing originally but i learnt about no-build-package the other day and figured it's a cleaner way to do what i want, but i may just switch back to doing it that way if i can't figure this out

[DetachHead]

i think i got it

1. git clone https://github.com/DetachHead/uv-issue-repro
2. cd uv-issue-repro
3. ./pw uv sync (using pyprojectx to pin the uv version, currently 0.5.20)
4. ./pw --lock (updates the pinned uv version to 0.5.24)
5. ./pw uv python pin 3.13 (previously 3.12.7)
6. ./pw uv sync --upgrade

[charliermarsh]

Great, thank you! I'll try again.

[charliermarsh]

I think the issue is that libcst is both a build dependency and a run-time dependency, and we're incorrectly sharing some resolution data between the two...