Unexpected (?) behavior when torch is a sub dependency #9498
Labels
bug
Something isn't working
Comments
|
I think this may be a duplicate of #9289 ? |
|
That's right -- we're actively working on it. Feel free to follow the linked issue. |
BurntSushi
added a commit
that referenced
this issue
Dec 10, 2024
This PR adds a notion of "conflict markers" to the lock file as an attempt to address #9289. The idea is to encode a new kind of boolean expression indicating how to choose dependencies based on which extras are activated. As an example of what conflict markers look like, consider one of the cases brought up in #9289, where `anyio` had unconditional dependencies on two different versions of `idna`. Now, those are gated by markers, like this: ```toml [[package]] name = "anyio" version = "4.3.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "idna", version = "3.5", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-project-foo'" }, { name = "idna", version = "3.6", source = { registry = "https://pypi.org/simple" }, marker = "extra == 'extra-7-project-bar' or extra != 'extra-7-project-foo'" }, { name = "sniffio" }, ] ``` The odd extra values like `extra-7-project-foo` are an encoding of not just the conflicting extra (`foo`) but also the package it's declared for (`project`). We need both bits of information because different packages may have the same extra name, even if they are completely unrelated. The `extra-` part is a prefix to distinguish it from groups (which, in this case, would be encoded as `group-7-project-foo` if `foo` were a dependency group). And the `7` part indicates the length of the package name which makes it possible to parse out the package and extra name from this encoding. (We don't actually utilize that property, but it seems like good sense to do it in case we do need to extra information from these markers.) While this preserves PEP 508 compatibility at a surface level, it does require utilizing this encoding scheme in order to evaluate them when they're present (which only occurs when conflicting extras/groups are declared). My sense is that the most complex part of this change is not just adding conflict markers, but their simplification. I tried to address this in the code comments and commit messages. Reviewers should look at this commit-by-commit. Fixes #9289, Fixes #9546, Fixes #9640, Fixes #9622, Fixes #9498, Fixes #9701, Fixes #9734
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I have this scenario:
I need a project that installs torch and torchvision ("torch==2.1.2", "torchvision==0.16.2") and a dependency (piq==0.8.0) that contains just one dependency: "torchvision>=0.10.0". I need to account for three different scenarios: Darwin OS (obv always without GPU) and Linux OS (both with and without GPU).
I am using the last version of uv (0.5.5).
Until the piq dependency enters the scene, everything works properly. But when I add piq (both if I add it to main dependencies or if I put it with CPU and GPU optional dependencies), it behaves unexpectedly adding both CPU and GPU versions of torch and torchvision (along with all heavy nvidia dependencies) even if i put --extra CPU:
and when searching in the lock, it can be seen that piq has as dependency both torchvision and torchvision+cpu:
Thanks in advance (:
The text was updated successfully, but these errors were encountered: