-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add an option to store virtual environments in a centralized location outside projects #1495
Comments
Thanks for the issue! This is the kind of thing we plan to tackle in the future, i.e. when we build an opinionated workflow for environment management. It's not in scope right this second, but we'll revisit it :) |
IMHO: Activating a venv is an antipattern because it introduces state in your shell. |
Imo there are various positive sideeffects of the venv not being local, although I dont personally like their chosen algorithm for selecting it. But ultimately, as long as the tool can know where the venv should be given the settings/invocation options, then there isn't generally a need to activate the venv, at least for |
I use Windows/Linux/Mac every day and synchronise my projects using OneDrive, having I'm using this plugin to manage and switch between different environments and don't have to know their location most of the time, but I do hope there is a standard for that. |
Note uv currently appears to work if you make If you don't have symlinks on your platform, this patch of uv may work for you by adding support for When uv does go in the higher level workflow direction, I'd advocate leaving a |
@hauntsaninja Thanks for the tip. I was annoyed that uv does not support the most common normal use-case of virtualenv ootb:
To make it work, the symlink indeed works
It would be nice if "uv venv" would make the link by default. And this would be a nice solution for managing venvs centrally outside the project-folder. @ResRipper and for syncing via onedrive, you can point .venv to one of the os-specific .venv-files
|
Just to add my voice that this would be really really nice to have. Different tools seem to choose either one approach or the other, and it would be great if For development situations where For other projects e.g. scientific ones that are in maybe shared or cloud or working folders, it's undesirable behaviour, and then working with My impression was that symlinks have poor portability, so like @DanCardin I'd prefer something like a simple ~/example/foo >>> uv venv -c
Using Python 3.11.9 interpreter at: /usr/bin/python3
Creating virtualenv at: /home/jdoe/.local/share/uv/example/foo/.venv
Activate with: uv venv activate
~/example/foo >>> uv venv activate
Activating virtualenv at /home/jdoe/.local/share/uv/example/foo/.venv
(foo) ~/example/foo >>> deactivate
~/example/foo >>> but with the addition that it would be cool to be able to also activate the venv by name from another location, with the search resolved intelligently and some ability to disambiguate; I could imagine that looking like: ~/some/folder >>> uv activate foo
Searching for virtualenvs at /home/jdoe/.local/share/uv/**/foo/.venv
Activating virtualenv at: /home/jdoe/.local/share/uv/example/foo/.venv
(foo) ~/some/folder >>> uv activate bar
Searching for virtualenvs at /home/jdoe/.local/share/uv/**/bar/.venv
error: virtualenv name is ambiguous! The following matches were found:
1) /home/jdoe/.local/share/uv/Documents/bar/.venv
2) /home/jdoe/.local/share/uv/project1/bar/.venv
disambiguate venvs with the same name using parents e.g. to activate 1) use:
uv venv activate Documents/bar
(foo) ~/some/folder >>> uv activate project1/bar
Searching for virtualenvs at /home/jdoe/.local/share/uv/**/project1/bar/.venv
Activating virtualenv at: /home/jdoe/.local/share/uv/project1/bar/.venv
(bar) ~/some/folder >>> |
So in addition to the "not wanting the venv to be stored in a folder that is backed up to the cloud" use case, I found another use case today:
(In my view this is a short-sighted approach on the tool's part, but it's just another example of why it might be necessary to keep a venv elsewhere.) |
For various reasons, I have a preference for out of tree virtual environments. Things just work if I symlink, but I don't know that this is guaranteed, so I thought I'd add a test for it. It looks like there's another code path that matters (`FoundInterpreter::discover -> PythonEnvironment::from_root`) for the higher level commands, but couldn't spot a good place to test that. Related discussion: #1495 (comment) / #1578 (comment)
…ONMENT` (#6834) Allows configuration of the (currently hard-coded) path to the virtual environment in projects using the `UV_PROJECT_ENVIRONMENT` environment variable. If empty, we'll ignore it. If a relative path, it will be resolved relative to the workspace root. If an absolute path, we'll use that. This feature targets use in Docker images and CI. The variable is intended to be set once in an isolated system and used for all uv operations. We do not expose a CLI option or configuration file setting — we may pursue those later but I see them as lower priority. I think a system-level environment variable addresses the most pressing use-cases here. This doesn't special-case the system environment. Which means that you can use this to write to the system Python environment. I would generally strongly recommend against doing so. The insightful comment from @edmorley at #5229 (comment) provides some context on why. More generally, `uv sync` will remove packages from the environment by default. This means that if the system environment contains any packages relevant to the operation of the system (that are not dependencies of your project), `uv sync` will break it. I'd only use this in Docker or CI, if anywhere. Virtual environments have lots of benefits, and it's only [one line to "activate" them](https://docs.astral.sh/uv/guides/integration/docker/#using-the-environment). If you are considering using this feature to use Docker bind mounts for developing in containers, I would highly recommend reading our [Docker container development documentation](https://docs.astral.sh/uv/guides/integration/docker/#developing-in-a-container) first. If the solutions there do not work for you, please open an issue describing your use-case and why. We do not read `VIRTUAL_ENV` and do not have plans to at this time. Reading `VIRTUAL_ENV` is high-risk, because users can easily leave an environment active and use the uv project interface today. Reading `VIRTUAL_ENV` would be a breaking change. Additionally, uv is intentionally moving away from the concept of "active environments" and I don't think syncing to an "active" environment is the right behavior while managing projects. I plan to add a warning if `VIRTUAL_ENV` is set, to avoid confusion in this area (see #6864). This does not directly enable centrally managed virtual environments. If you set `UV_PROJECT_ENVIRONMENT` to an absolute path and use it across multiple projects, they will clobber each other's environments. However, you could use this with something like `direnv` to achieve "centrally managed" environments. I intend to build a prototype of this eventually. See #1495 for more details on this use-case. Lots of discussion about this feature in: - astral-sh/rye#371 - astral-sh/rye#1222 - astral-sh/rye#1211 - #5229 - #6669 - #6612 Follow-ups: - #6835 - #6864 - Document this in the project concept documentation (can probably re-use some of this post) Closes #6669 Closes #5229 Closes #6612
and
Even this makes things more complex than needed if you sync to the cloud across multiple systems, because the link might need to be to different places on different machines. If only for compatibility and ease of migration from one tool to another, I would offer the possibility to do what pdm does. That tool offers either the option to have a local Note that having the venvs all stored in a single place also makes it easier to apply deduplication tools on suitable filesystems. |
@zanieb is this a feature you're considering for the short term, or it's not high priority? |
Just to weight in, I sometimes edit projects (mainly for college) off a SMB share on my home server. I generally prefer having |
@Cornelius-Figgle Does a symlink work for you, e.g., |
I'm long-time I wrote a small bash/zsh script called uv-virtualenvwrapper that provides core virtualenvwrapper-like functionality for uv with tab completion. |
Just hit the need for multiple as I have a project that I test on both Windows and WSL. avasam@Horus:/mnt/e/Users/Avasam/Documents/Git/AutoSplit$ uv venv .venv-linux
Using CPython 3.11.9 interpreter at: /home/avasam/.pyenv/versions/3.11.9/bin/python3
Creating virtual environment at: .venv-linux
Activate with: source .venv-linux/bin/activate
avasam@Horus:/mnt/e/Users/Avasam/Documents/Git/AutoSplit$ source .venv-linux/bin/activate
(.venv-linux) avasam@Horus:/mnt/e/Users/Avasam/Documents/Git/AutoSplit$ uv sync
warning: `VIRTUAL_ENV=.venv-linux` does not match the project environment path `.venv` and will be ignored
error: Project virtual environment directory `/mnt/e/Users/Avasam/Documents/Git/AutoSplit/.venv` cannot be used because it is not a valid Python environment (no Python executable was found) Edit: See answer below. Not a full out-of-the-box solution. But it works as a workaround. One can use |
Correct. You can use |
If uv would gain this functionality, I would like the option to easily create a .venv-symlink to that centralized venv-location outside the project directory. |
Are there any updates on plans to support this feature? A natively supported interface similar to virtualenvwrapper would be incredible. I think it's a pretty common desire to not want a .venv folder to be included in OneDrive/iCloud file syncing. The ability to re-use environments across different folders would be a huge plus as well. |
Using the |
I had an idea about this issue. To solve the conundrum, why not have an option to store the Every time we run I believe this would not introduce any issue? |
To provide some more context to this idea: However, right now, if we are unable to store the What I propose above, to have an env variable |
I've tried this approach for a bit. I have my own little shell/batch script that manages my "central" venvs for me, on Windows, Mac, WSL, and HPC. One issue this introduces is that Git does not follow symlinks, so it won't see the |
indeed, you'd have to add a .gitignore entry to the gitignore in the root directory in that case... |
In the discussion above, @ResRipper and @matterhorn103 talked about one motivation for having the virtual environment outisde the project directory being to stop cloud sync tools from uploading the If I've read the above discussion correctly, one suggestion has been to place a symlink at I've noticed this behaviour when using detached environments with |
Maybe for your case it would be nice if uv could support either |
Most existing tooling doesn't care about the path of your venv as it is, they look for
but i'm somewhat pessimistic on whether any of this will get dealt with in any case, due to their stance on respecting VIRTUAL_ENV in the first place...
per my OP, I would very much prefer a solution that allows me to have multiple named venvs per project, typically for testing multiple versions of python in quick sequence). so while it doesnt need to be my suggestion, i think |
If you use bash, you can setup a command that looks for virtual environments at a global location (e.g. aa() {
if [[ $# -eq 0 ]]
then
. .venv/bin/activate
else
. ~/.venvs/$1/bin/activate
fi
}
export -f aa
_activate_venv_autocomplete() {
if [[ $COMP_CWORD == 1 ]]
then
local word=${COMP_WORDS[COMP_CWORD]}
local envs=$(ls -1 $HOME/.venvs)
COMPREPLY=($(compgen -W "$envs" -- "$word"))
else
COMPREPLY=()
fi
}
complete -F _activate_venv_autocomplete aa When you create a new env with
|
My point is that uv’s mental model is to deprecate activating environments, as it can lead to problems and mismatches. I would like to be able to use uv’s workflow in setups where I normally can’t, such as HPC clusters. |
I think we are very far from being able to deprecate activating environments. Might as well make my current life easier 😉 |
Off topic
I don't think so. |
Off topic
Unfortunately, technology changes faster than (most) people. I already use |
Off topicIt is a matter of what you are used to and what you are doing. Yes, in many cases not having the shell hold the a state can be advantageous. Yet, depending on what you do typing |
I tried setting This mostly works but I'm one issue I'm having is that |
I ideally dont .venv/ folders cluttering my project folders (not the least because they may not be safe to move, at least with
venv
)In my personal (also rust) workflow tool (that i'd love to not have to maintain if
uv
could arrive at some of the same decisions 😆) there is a setting which when set to "central" puts all venvs in$XDG_DATA_HOME/<toolname>/...
, and the path to the venv is determined by mirroring the path to the project. That is,~/foo/bar/baz/pyproject.toml
->$XDG_DATA_HOME/uv/foo/bar/baz/.venv/
.By comparison to more traditional tools,
poetry
also (by default) will automatically create venvs in a central location. Although it puts all venvs in the same folder using a somewhat inscrutable hash to disambiguate projects of the same name.This ^ feature sort of implies a few other mostly separate features in order to be useful:
uv activate
(because it becomes impractical to self-activate, except by copy-pasting the path that gets printed)uv self init --shell zsh
(or whatever), that gives you the shell integration to automatically activate through the CLI itself.uv venv --delete
or-d
The text was updated successfully, but these errors were encountered: