Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

uv sync always tries to resolve all optional-dependencies #6729

Closed
chrisjsewell opened this issue Aug 27, 2024 · 12 comments · Fixed by #6982
Closed

uv sync always tries to resolve all optional-dependencies #6729

chrisjsewell opened this issue Aug 27, 2024 · 12 comments · Fixed by #6982
Assignees
Labels
documentation Improvements or additions to documentation

Comments

@chrisjsewell
Copy link

chrisjsewell commented Aug 27, 2024

the initial options of uv sync would lead me to believe that extras should not be resolved by default

$ uv sync --help       
Update the project's environment

Usage: uv sync [OPTIONS]

Options:
      --extra <EXTRA>                            Include optional dependencies from the extra group name
      --all-extras                               Include all optional dependencies

yet when I try to run it, I get e.g:

$ uv sync
× No solution found when resolving dependencies:
...
because myproject[docs] depends on projectx ...

It's weird, because if I remove projectx from the doc optional-dependencies, it works and none of the other dependencies in doc are included in the .venv


I see this is because uv tries to add all optional-dependencies to the uv.lock first, but what if I do not want them in there, because they have incompatibilities

@charliermarsh
Copy link
Member

This is by design. We resolve the entire project at once, but would only install the subset that's relevant. Conflicting optional dependencies are not yet supported.

@chrisjsewell
Copy link
Author

chrisjsewell commented Aug 27, 2024

Conflicting optional dependencies are not yet supported

heya, yeh this is a blocker for using uv in a similar way to e.g. tox, i.e. having different optional groups, for running tests, doc builds, etc against different dependency matrices (that can potentially conflict).

I guess hopefully #5632 should allow this one day

but for now I'll probably stick to https://github.com/tox-dev/tox-uv (which works better for this) rather than "full-blown" uv 😅

@zanieb zanieb self-assigned this Aug 27, 2024
@zanieb zanieb added the documentation Improvements or additions to documentation label Aug 27, 2024
@zanieb
Copy link
Member

zanieb commented Aug 27, 2024

We should document this in https://docs.astral.sh/uv/concepts/projects/#project-lockfile

I guess we should also have an issue specifically tracking allowing conflicting extras.

I'll follow up on these things.

@chrisjsewell
Copy link
Author

cheers

@chrisjsewell
Copy link
Author

To clarify, the main use case, is that I want to be able to do something like this: tox -e py38-sphinx6,py310-sphinx8,
i.e. running pytest against different versions of Python, and potentially different versions of key "peer dependencies"

@arsatiki
Copy link

arsatiki commented Sep 3, 2024

I was bit by this today as well, trying to sync the mlflow repository. The --help options seemed contradictory, since I was very emphatically not installing any extras and it still croaked on jfrog 😒

@zanieb
Copy link
Member

zanieb commented Sep 3, 2024

@arsatiki what do you expect the --help output to say?

@zanieb
Copy link
Member

zanieb commented Sep 3, 2024

Tracking support in #6981

@arsatiki
Copy link

arsatiki commented Sep 4, 2024

@zanieb I think the help texts for one or both options should mention that they are still trying to resolve all extras, even if they are not installed.

At least I do not find it intuitive that an extra would affect the resolution, unless I specifically opt into the extra. (But I get why you have chosen the opposite strategy.) Since it does the (in my view) unintuitive thing, the relevant option should notify me about it.

@zanieb
Copy link
Member

zanieb commented Sep 4, 2024

Thanks I'll update that.

@zanieb
Copy link
Member

zanieb commented Sep 4, 2024

See #7013

BurntSushi added a commit that referenced this issue Nov 13, 2024
This PR adds support for conflicting extras. For example, consider
some optional dependencies like this:

```toml
[project.optional-dependencies]
project1 = ["numpy==1.26.3"]
project2 = ["numpy==1.26.4"]
```

These dependency specifications are not compatible with one another.
And if you ask uv to lock these, you'll get an unresolvable error.

With this PR, you can now add this to your `pyproject.toml` to get
around this:

```toml
[tool.uv]
conflicting-groups = [
    [
      { package = "project", extra = "project1" },
      { package = "project", extra = "project2" },
    ],
]
```

This will make the universal resolver create additional forks
internally that keep the dependencies from the `project1` and
`project2` extras separate. And we make all of this work by reporting
an error at **install** time if one tries to install with two or more
extras that have been declared as conflicting. (If we didn't do this,
it would be possible to try and install two different versions of the
same package into the same environment.)

This PR does *not* add support for conflicting **groups**, but it is
intended to add support in a follow-up PR.

Closes #6981

Fixes #8024

Ref #6729, Ref #6830

This should also hopefully unblock
dagster-io/dagster#23814, but in my testing, I
did run into other problems (specifically, with `pywin`). But it does
resolve the problem with incompatible dependencies in two different
extras once you declare `test-airflow-1` and `test-airflow-2` as
conflicting for `dagster-airflow`.

NOTE: This PR doesn't make `conflicting-groups` public yet. And in a
follow-up PR, I plan to switch the name to `conflicts` instead of
`conflicting-groups`, since it will be able to accept conflicting extras
_and_ conflicting groups.
@wimiko
Copy link

wimiko commented Jan 29, 2025

I have a variation of this problem, but not so much with conflicting extras, but rather with missing extras:

[project.optional-dependencies]
with_everything = [
    "something-extra",
]

[tool.uv.sources]
something-extra = { path = "../something_extra" }

I was hoping that I would be able to do uv run without the extras when "../something_extra" doesn't exist, but that doesn't seen to be the case:

error: Distribution not found at: file:///...

I also tried adding dependency-metadata for this package in the hope that it wouldn't try to find it, but that didn't help.

I'm guessing that the solution proposed in #8976 doesn't really help in this situation?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants