uv lock Improvement: a consistent environment for prepare_metadata_for_build_wheel.

[YouJiacheng]

Currently, uv lock runs in the Python interpreter specified by "Python options" (shown by uv lock --help).

However, as discussed in #7291 , even managed Python Interpreters can have an environment inconsistent with the default virtual environment created by uv venv.
Let uv lock be "polluted" by the system Python Interpreter with an arbitrary environment sounds even worse.

[deleted]
Ideally we can specify this in pyproject.toml with a default to an environment equivalent to the one created by uv venv.
And we can also specify index-url.
Expectation by example:

[tool.uv]
lock-dependencies-default = []

[tool.uv.lock-index-url]
torch = "https://download.pytorch.org/whl/cu124"

[tool.uv.lock-dependencies]
flash-attn = ["setuptools", "packaging"]
chumpy = ["setuptools", "pip"]

OR

[[tool.uv.lock-resolution]]
name = "torch"
index-url = "https://download.pytorch.org/whl/cu124"
dependencies = ["setuptools"]

[[tool.uv.lock-resolution]]
name = "flash-attn"
dependencies = ["setuptools", "packaging"]

[[tool.uv.lock-resolution]]
name = "chumpy"
dependencies = ["setuptools", "pip"]

[deleted]

[charliermarsh]

There's a significant downside to this, which is that it would require that we create a virtual environment to perform uv lock (despite the fact that we never modify it). I'm hesitant to make that change, given that (1) build isolation is relatively rare, and (2) we want to redesign it anyway.

Can you file the second suggestion here as its own issue? I think it's separate.

\cc @zanieb

[YouJiacheng]

"the second suggestion here" -- did you mean index-url stuffs or the whole "lock resolution config" thing?

[YouJiacheng]

IIUC all package that have a dynamic metadata or without a pyproject.toml will need an environment to run prepare_metadata_for_build_wheel?

[YouJiacheng]

we want to redesign it anyway.

is there any roadmap for this?

[charliermarsh]

I misread your second suggestion. I thought you were suggesting something like #7393 which would also fix this by allowing you to define the Chumpy requirements upfront rather than requiring that we build it.

[charliermarsh]

The [tool.uv.lock-index-url] is unlikely to happen as we're adding an entirely separate API for indexes (#171).

[charliermarsh]

If a user said:

[[tool.uv.lock-resolution]]
name = "flash-attn"
dependencies = ["setuptools", "packaging", "torch"]

Then we would install setuptools, packaging, and torch in an isolated environment -- is that the suggestion?

I don't know that this would solve the general case. It's typically not the case that users disable build isolation because a package fails to declare the right build dependencies -- that is sometimes true, like for Chumpy, but typically, build isolation is disabled because you need to build against a specific version of an installed dependency, like for PyTorch.

[YouJiacheng]

Okay that sounds bad.
Useful cases are too rare.
And this can confuse the users -- "is it the dependencies for setup.py of the package or the whole package?" -- even you misread my example.

[charliermarsh]

is there any roadmap for this?

Let me look... The main idea was that we wanted to add a new build isolation API for uv sync and uv lock, but we didn't have time so we decided to support --no-build-isolation like in pip to start. I have some unformed ideas around allowing packages to declare the dependencies that they need to pull in from the existing environment, kind of like what you had above... E.g.:

name = "flash-attn"
dependencies = ["setuptools", "packaging", "torch"]

...could create a virtual environment, and then give it access to the already-installed torch from another virtual environment. So it'd be a sort of hybrid form of isolation: the build happens in a new, isolated environment, but we link in packages that are installed in a separate environment.

[YouJiacheng]

I think flash-attn can be solved by #7393 + switching between two optional dependencies groups?
But yeah "hybrid form of isolation" is not bad.

[YouJiacheng]

Can we provide a first-class support for "switching between two (or more) optional dependencies groups"?
Just standardize it with inspiration from Docker multi-stage builds -- the outputs of build stages are wheels.

[[tool.uv.build-stage]]
# environment variables etc. can also be set.
dependencies = ["setuptools", "packaging", "psutil", "ninja", "torch"]
outputs = ["flash-attn", "some-other-packages"]
[[tool.uv.build-stage]]
dependencies = ["setuptools", "torch", "flash-attn"]
outputs = ["some-packages"]