Add an option to store virtual environments in a centralized location outside projects

[DanCardin]

I ideally dont .venv/ folders cluttering my project folders (not the least because they may not be safe to move, at least with venv)

In my personal (also rust) workflow tool (that i'd love to not have to maintain if uv could arrive at some of the same decisions 😆) there is a setting which when set to "central" puts all venvs in $XDG_DATA_HOME/<toolname>/..., and the path to the venv is determined by mirroring the path to the project. That is, ~/foo/bar/baz/pyproject.toml -> $XDG_DATA_HOME/uv/foo/bar/baz/.venv/.

By comparison to more traditional tools, poetry also (by default) will automatically create venvs in a central location. Although it puts all venvs in the same folder using a somewhat inscrutable hash to disambiguate projects of the same name.

---

This ^ feature sort of implies a few other mostly separate features in order to be useful:

• uv activate (because it becomes impractical to self-activate, except by copy-pasting the path that gets printed)
• which then implies some uv self init --shell zsh (or whatever), that gives you the shell integration to automatically activate through the CLI itself.
• uv venv --delete or -d

[zanieb]

Thanks for the issue!

This is the kind of thing we plan to tackle in the future, i.e. when we build an opinionated workflow for environment management. It's not in scope right this second, but we'll revisit it :)

[woutervh]

By comparison to more traditional tools, poetry also (by default) will automatically create venvs in a central location. Although it puts all venvs in the same folder using a somewhat inscrutable hash to disambiguate projects of the same name.

IMHO:
It's one of the really bad decisions of poetry.
As a python contractor usually supporting other python developers, I really dislike any situation where devs have no idea where their virtualenv of their project is. Your executables are the most important thing in your project, hiding them in arcane directories should never be a default.

Activating a venv is an antipattern because it introduces state in your shell.
I really hope it can be de-emphasized in the future.

[DanCardin]

Imo there are various positive sideeffects of the venv not being local, although I dont personally like their chosen algorithm for selecting it.

But ultimately, as long as the tool can know where the venv should be given the settings/invocation options, then there isn't generally a need to activate the venv, at least for uv itself to function. But it's the reality that many tools require VIRTUAL_ENV to be set to function properly, which either means activating the venv or routing all commands through a uv run (which might be nice interactively but isn't ideal for committed files imo).

[ResRipper]

I use Windows/Linux/Mac every day and synchronise my projects using OneDrive, having .venv in my project folder is such a nightmare, and that's why I started using pipenv. IMO venv should always be centralised as environments and packages are platform dependent, they are not part of the "content" of the project.

I'm using this plugin to manage and switch between different environments and don't have to know their location most of the time, but I do hope there is a standard for that.

[hauntsaninja]

Note uv currently appears to work if you make .venv file a symlink to your actual venv

If you don't have symlinks on your platform, this patch of uv may work for you by adding support for .venv files that point to the actual venv location: #1578 (comment)

When uv does go in the higher level workflow direction, I'd advocate leaving a .venv symlink or file pointing to the central location. This makes it easy for IDEs and other tools to figure out where the environment is. This .venv file / symlink idea was discussed around the time PEP 704 was a thing and had fairly positive support

[woutervh]

@hauntsaninja Thanks for the tip.

I was annoyed that uv does not support the most common normal use-case of virtualenv ootb:

> virtualenv foo
> cd foo
> bin/pip install <package1>

To make it work, the symlink indeed works

> uv venv foo
> cd foo
> ln -s .  .env
> uv pip install <package2>

It would be nice if "uv venv" would make the link by default.

And this would be a nice solution for managing venvs centrally outside the project-folder.

@ResRipper and for syncing via onedrive, you can point .venv to one of the os-specific .venv-files

.venv -> .venv-linux
.venv-linux  --> /some/linux/path
.venv-mac  --> /some/mac/path
.venv-windows  --> /some/windows/path

[matterhorn103]

Just to add my voice that this would be really really nice to have. Different tools seem to choose either one approach or the other, and it would be great if uv would do both as I like and use both in different situations:

For development situations where git is being used, having .venv there in the project's folder is convenient.

For other projects e.g. scientific ones that are in maybe shared or cloud or working folders, it's undesirable behaviour, and then working with conda is much smoother than with Python venvs. That's pretty much the only thing that keeps me using conda for some things (other than the different ecosystem, naturally).

My impression was that symlinks have poor portability, so like @DanCardin I'd prefer something like a simple -c switch to create the venv in an automatically determined central location, and simple automatic activation, e.g.:

~/example/foo >>> uv venv -c
Using Python 3.11.9 interpreter at: /usr/bin/python3
Creating virtualenv at: /home/jdoe/.local/share/uv/example/foo/.venv
Activate with: uv venv activate
~/example/foo >>> uv venv activate
Activating virtualenv at /home/jdoe/.local/share/uv/example/foo/.venv
(foo) ~/example/foo >>> deactivate
~/example/foo >>>

but with the addition that it would be cool to be able to also activate the venv by name from another location, with the search resolved intelligently and some ability to disambiguate; I could imagine that looking like:

~/some/folder >>> uv activate foo
Searching for virtualenvs at /home/jdoe/.local/share/uv/**/foo/.venv
Activating virtualenv at: /home/jdoe/.local/share/uv/example/foo/.venv
(foo) ~/some/folder >>> uv activate bar
Searching for virtualenvs at /home/jdoe/.local/share/uv/**/bar/.venv
error: virtualenv name is ambiguous! The following matches were found:
  1) /home/jdoe/.local/share/uv/Documents/bar/.venv
  2) /home/jdoe/.local/share/uv/project1/bar/.venv
disambiguate venvs with the same name using parents e.g. to activate 1) use:
  uv venv activate Documents/bar
(foo) ~/some/folder >>> uv activate project1/bar
Searching for virtualenvs at /home/jdoe/.local/share/uv/**/project1/bar/.venv
Activating virtualenv at: /home/jdoe/.local/share/uv/project1/bar/.venv
(bar) ~/some/folder >>>

[matterhorn103]

So in addition to the "not wanting the venv to be stored in a folder that is backed up to the cloud" use case, I found another use case today:

• Creating compiled binaries of Qt apps written with PySide using pyside6-deploy (a nuitka wrapper) isn't possible if a venv folder is present in the application folder

(In my view this is a short-sighted approach on the tool's part, but it's just another example of why it might be necessary to keep a venv elsewhere.)

[callegar]

When uv does go in the higher level workflow direction, I'd advocate leaving a .venv symlink or file pointing to the central location.

and

It would be nice if "uv venv" would make the link by default.
And this would be a nice solution for managing venvs centrally outside the project-folder.

Even this makes things more complex than needed if you sync to the cloud across multiple systems, because the link might need to be to different places on different machines.

If only for compatibility and ease of migration from one tool to another, I would offer the possibility to do what pdm does. That tool offers either the option to have a local .venv or the option to have venvs stored all together in a centralized place.

Note that having the venvs all stored in a single place also makes it easier to apply deduplication tools on suitable filesystems.

[PhilipVinc]

@zanieb is this a feature you're considering for the short term, or it's not high priority?

[Cornelius-Figgle]

Just to weight in, I sometimes edit projects (mainly for college) off a SMB share on my home server. I generally prefer having .venv in the project folder as it is neater, but the venv tools for python don't seem to play nice with SMB, so the solution is to store the venvs locally. In poetry this is rather easy as you just add a poetry.toml to the project that sets in-project to false (since my global config sets it to true for most projects). Having a way to do this in uv would be very useful, as besides this uv seems an amazing tool.

[zanieb]

@Cornelius-Figgle Does a symlink work for you, e.g., .venv -> <somewhere else>?

[sitic]

I'm long-time virtualenvwrapper user and miss the convenience of activating central, named, virtual environments with a quick workon command.

I wrote a small bash/zsh script called uv-virtualenvwrapper that provides core virtualenvwrapper-like functionality for uv with tab completion. mkvirtualenv <name> to create (arguments passed to uv venv), workon <name> to activate, rmvirtualenv to remove, and lsvirtualenv to list environments stored. Virtual environments are stored in ~/.virtualenvs by default and the script compatible with existing virtualenvwrapper venvs.

[Avasam]

Just hit the need for multiple as I have a project that I test on both Windows and WSL.
I thought I could simply manually activate the .venv first to get uv to use it. But uv actively denies respecting VIRTUAL_ENV:

avasam@Horus:/mnt/e/Users/Avasam/Documents/Git/AutoSplit$ uv venv .venv-linux
Using CPython 3.11.9 interpreter at: /home/avasam/.pyenv/versions/3.11.9/bin/python3
Creating virtual environment at: .venv-linux
Activate with: source .venv-linux/bin/activate
avasam@Horus:/mnt/e/Users/Avasam/Documents/Git/AutoSplit$ source .venv-linux/bin/activate
(.venv-linux) avasam@Horus:/mnt/e/Users/Avasam/Documents/Git/AutoSplit$ uv sync
warning: `VIRTUAL_ENV=.venv-linux` does not match the project environment path `.venv` and will be ignored
error: Project virtual environment directory `/mnt/e/Users/Avasam/Documents/Git/AutoSplit/.venv` cannot be used because it is not a valid Python environment (no Python executable was found)

Edit: See answer below. Not a full out-of-the-box solution. But it works as a workaround. One can use pyenv to manage different environments and manually update UV_PROJECT_ENVIRONMENT if using WSL for multiple projects.

[charliermarsh]

Correct. You can use UV_PROJECT_ENVIRONMENT if you want to overwrite it, per the docs: https://docs.astral.sh/uv/concepts/projects/config/#project-environment-path.

[woutervh]

If uv would gain this functionality, I would like the option to easily create a .venv-symlink to that centralized venv-location outside the project directory.

[Nick-Hemenway]

Are there any updates on plans to support this feature? A natively supported interface similar to virtualenvwrapper would be incredible. I think it's a pretty common desire to not want a .venv folder to be included in OneDrive/iCloud file syncing. The ability to re-use environments across different folders would be a huge plus as well.

[Avasam]

For me, uv 0.5.29's --active flag resolved my need for this feature for testing though WSL:
I can manually create an environment once. Then as long as that env is activated, uv sync --activated will work to update my dependencies when in WSL.

[Nick-Hemenway]

Using the --active flag works and allows a user to add and remove packages to a centralized virtual environment using, e.g. uv add --active numpy or uv remove --active. It would be nice if there was a way (perhaps through an environment variable?) to turn the --active flag on by default so that one doesn't have to remember to type the --active flag on every time they invoke uv to update their package/environment dependencies.

[PhilipVinc]

I had an idea about this issue.
If I understand correctly, a counter-argument to storing virtual environments in a centralised location are other tools would not work correctly because they do not know where to look for the venv directory.
However, there are several situations where this is necessary (again, storing projects in dropbox/cloud services, or working in HPC environments where projects and virtual environments should be stored on different filesystems).

To solve the conundrum, why not have an option to store the .venv directory in a centralised location, as to address this issue, but symlink it to the current project directory, as to allow discoverability by all other tools?

Every time we run uv sync --depot XYZ or ùv run --depot XZY the symlink would be recreated if stale.

I believe this would not introduce any issue?

[PhilipVinc]

To provide some more context to this idea:
I fully buy in uv's point of view that activating environments is an anti-pattern and that the best approach is never to have to do that.

However, right now, if we are unable to store the .venv in the current directory for whatever reason (again, HPC setups usually forbid or prevent this, or cloud setups...) the current workaround, of declaring VIRTUAL_ENV + --active or UV_PROJECT_ENVIRONMENT is equivalent to activating an environment.

What I propose above, to have an env variable UV_GLOBAL_DEPOT=XYZ which automatically stores the .venv in the global depot. The precise path could be something like $UV_GLOBAL_DEPOT/$(hash project_path)
However, by simlinking the venv to the project directory, the fact that the venv is stored elsewhere becomes unimportant for all existing tooling.
This would allow to benefit from uv's opinionated 'no activation workflow' on systems where the current behaviour cannot be used.

[john-hen]

I believe this would not introduce any issue?

I've tried this approach for a bit. I have my own little shell/batch script that manages my "central" venvs for me, on Windows, Mac, WSL, and HPC. One issue this introduces is that Git does not follow symlinks, so it won't see the .gitignore file that UV places there for our convenience. Plus, creating symlinks on Windows (usually) requires admin privileges, so UV would need to learn how to request a UAC prompt.

[PhilipVinc]

indeed, you'd have to add a .gitignore entry to the gitignore in the root directory in that case...

[jatonline]

@PhilipVinc

I believe this would not introduce any issue?

In the discussion above, @ResRipper and @matterhorn103 talked about one motivation for having the virtual environment outisde the project directory being to stop cloud sync tools from uploading the .venv to the cloud.

If I've read the above discussion correctly, one suggestion has been to place a symlink at .venv to some central depot directory. I'm not sure how well-known this is, so thought it might be helpful to add that: on some systems (e.g. recent MacOS and Dropbox), if you put symlink inside the cloud sync folder, this gets followed by the sync client and so get uploaded to the cloud as well. (So I'm thinking this might not fix some of the originally stated issues?)

I've noticed this behaviour when using detached environments with pixi, too.

[callegar]

Maybe for your case it would be nice if uv could support either .venv be the virtual environment directory or it be a file (I think that git does something similar on platforms that do not support symlinks well). In the latter case, it could contain a line such as linkto = <location>. So it could work equivalently to a symlink, with the additional advantages of: (i) not being incorrectly followed by poor synchronization tools; and (ii) being easier on platforms where symlinks are not so well supported. Furthermore, this could be enhanced by letting <location> be a string where some keyworks get expanded. For instance {hostname}, {project_name}, {path_hash} (being the hash of the project path), or {python_version}. This would let one easily avoid clashes (e.g. similarly to what pdm does for centralized venvs).

[DanCardin]

However, by simlinking the venv to the project directory, the fact that the venv is stored elsewhere becomes unimportant for all existing tooling.

Most existing tooling doesn't care about the path of your venv as it is, they look for VIRTUAL_ENV. So, i think all all a symlink does is work around the lack of a uv activate command (and corresponding shell integration) which sets that env var to the central location determined by uv.

uv's preference against venv activation makes sense certainly for uv commands themselves; i think it starts to get more annoying when i'm forced to uv run pytest instead of just pytest; and i think it devolves from there with e.g. vim where it needs to be in the env for editor tooling to function (or else be able to detect the venv a-la vscode).

but i'm somewhat pessimistic on whether any of this will get dealt with in any case, due to their stance on respecting VIRTUAL_ENV in the first place...

$UV_GLOBAL_DEPOT/$(hash project_path)

per my OP, I would very much prefer a solution that allows me to have multiple named venvs per project, typically for testing multiple versions of python in quick sequence).

so while it doesnt need to be my suggestion, i think $some-root-uv-path/relative/path/to/project/{venv-name} (where venv-name defaults to .venv.{python-version}) makes a lot of sense, such that uv run -p 3.11 "just works" and selects the correct venv for the selected version of python.

[francois-rozet]

If you use bash, you can setup a command that looks for virtual environments at a global location (e.g. ~/.venvs) and activates them based on the env name. For example, copy-paste the following in your .bashrc and now (after restarting the shell) the aa command allows to activate a globally stored env, with auto-completion!

aa() {
    if [[ $# -eq 0 ]]
    then
        . .venv/bin/activate
    else
        . ~/.venvs/$1/bin/activate
    fi
}

export -f aa

_activate_venv_autocomplete() {
    if [[ $COMP_CWORD == 1 ]]
    then
        local word=${COMP_WORDS[COMP_CWORD]}
        local envs=$(ls -1 $HOME/.venvs)
        COMPREPLY=($(compgen -W "$envs" -- "$word"))
    else
        COMPREPLY=()
    fi
}

complete -F _activate_venv_autocomplete aa

When you create a new env with uv, remember to store it in ~/.venvs.

uv venv ~/.venvs/myenv
aa myenv

[PhilipVinc]

My point is that uv’s mental model is to deprecate activating environments, as it can lead to problems and mismatches.

I would like to be able to use uv’s workflow in setups where I normally can’t, such as HPC clusters.
Right now it’s impossible.

[francois-rozet]

I think we are very far from being able to deprecate activating environments. Might as well make my current life easier 😉

[astrojuanlu]

Off topic

I think we are very far from being able to deprecate activating environments.

I don't think so. uv run already works and takes care of everything under the hood. It's an abstraction, so in the future it could decouple itself from in-tree venvs. Or somebody else could create a uvc run that uses centralized locations. The point is to stop doing source .venv/bin/activate or whatever OS-specific command you need (see also #1910). In Node.JS and Rust lands folks are used to doing npm run and cargo run, I don't see why we can't adopt that in Python. For that though, people need to internalise this abstraction, and documentation and instructional materials need to be updated accordingly.

[francois-rozet]

Off topic

For that though, people need to internalise this abstraction, and documentation and instructional materials need to be updated accordingly.

Unfortunately, technology changes faster than (most) people. I already use uv run, it is great. But I also know people that are still using Python 2.7 and 3.6. And on HPC clusters, the norm is still (and will likely remain for a while) to create an env, activate it, install your dependencies and run your code. Improving that workflow with uv venv and uv pip is already a huge step. Deprecating activating the env can come later.

[callegar]

Off topic

It is a matter of what you are used to and what you are doing. Yes, in many cases not having the shell hold the a state can be advantageous. Yet, depending on what you do typing uv run every time might end up more verbose than typing uv venv --activate once (should anything like this get supported or . .venv/bin/activate otherwise). Furthermore, there are really cases when you want to use uv, not to develop a project, but just to have a well defined virtual environment to play in (think working on notebooks that need a specific environment). In the latter case, you may really prefer having the shell holding the state to assure that you can cd to different places maintaining the state.

[mil-ad]

I tried setting UV_PROJECT_ENVIRONMENT to something like export UV_PROJECT_ENVIRONMENT=/scratch/$USER/venvs/${PWD#$HOME}" for all users.

This mostly works but I'm one issue I'm having is that uv venv ignores UV_PROJECT_ENVIRONMENT #10807