Detect venv set to $VIRTUAL_ENV

[pantheraleo-7]

Setting a path to a venv to $VIRTUAL_ENV is a common way to signify a "global" venv, which should be used if a project-level venv is not present. This is especially true after system level python installs have become externally managed PEP668 on almost all platforms (debian, ubuntu, brew, etc.)

[pantheraleo-7]

This is different from when you activate a venv and it automatically sets this variable to itself. Am talking about a user setting this variable in their dotfiles.

One could put this "global" venv on the $PATH but this leads to various undesirable side-effects in user experience.

[brettcannon]

Setting a path to a venv to $VIRTUAL_ENV is a common way to signify a "global" venv, which should be used if a project-level venv is not present.

I wouldn't say it's "common", but is a hack some people have used to work around code which checks that variable as a way to determine if a virtual environment is set. But as the docs point out, that environment variable is only meant to signify when a virtual environment is activated and not as a way to tell other apps, "use this system install".

We also wouldn't want to promote working around externally-managed Python installs as the OS vendors set that specifically because they don't want users installing into them and some tools assume virtual environments can bn installed into.

[pantheraleo-7]

"use this system install"
We also wouldn't want to promote working around externally-managed Python installs as the OS vendors set that specifically because they don't want users installing into them

Where did system installs or OS pythons come from? I am specifically talking about a venv here.

From the docs:

You don’t specifically need to activate a virtual environment, as you can just specify the full path to that environment’s Python interpreter when invoking Python. Furthermore, all scripts installed in the environment should be runnable without activating it.

As you can see, activating a venv isn't mandated.

You surely are not saying that a large amount of use cases (data science, etc) don't require the presence of a global venv (pyenv, etc install their venvs to a central location for a reason), are you? For tools that don't have a central location for venvs like uv, setting a venv to this env var becomes critical to the workflow. Also, uv (which strives strictly to abide by PEPs) uses this env var to locate venv, why can't pet?

[pantheraleo-7]

Also, just fyi, setting an externally-managed installation to this env var won't make it not externally managed xD

Even if it did, pet has got nothing to do with that use case because it is supposed to locate installations (which it already does for externally managed installs anyway) and not install packages into it. pet is not pip.

[pantheraleo-7]

Just like pet tries to locate venvs in various places like the workspace itself, many central locations used by tools, user paths like XDG_BIN_HOME, the $PATH env var, etc. pet should also use this env var to locate venvs. And this is exactly what the doc says:

When a virtual environment has been activated, the VIRTUAL_ENV environment variable is set to the path of the environment. Since explicitly activating a virtual environment is not required to use it, VIRTUAL_ENV cannot be relied upon to determine whether a virtual environment is being used.

It says "use this env var to locate venvs except when it is not set for some reason". Currently, pet doesn't even check this env var to begin with. If anything, pet is going against the docs here.

[brettcannon]

Where did system installs or OS pythons come from?

Sorry, misread '"global" venv' as '"global" env'.

As you can see, activating a venv isn't mandated.

I'm aware as I am the one who opened the issue on CPython to make that abundantly clear in the docs.

You surely are not saying that a large amount of use cases (data science, etc) don't require the presence of a global venv (pyenv, etc install their venvs to a central location for a reason), are you?

What does "global" mean to you in this instance? A virtual environment that is shared across multiple projects or a virtual environment that's meant to be used for a single project, but having it stored outside of the project directory?

uv (which strives strictly to abide by PEPs)

As someone who wrote a bunch of those PEPs, I'm acutely aware of what uv does and does not implement based on specs, convention, and what they just chose to do simply because they thought it was a good idea.

uses this env var to locate venv, why can't pet?

It could. I'm just pushing back on your assertion that people manually setting VIRTUAL_ENV is "common" when not fully activating a virtual environment (i.e. updating PATH).

[pantheraleo-7]

Let me explain my case.

In fields like data science, a global environment with a certain set of packages are always needed for quick prototyping, running notebooks, etc. (imagine a bunch of tightly nit packages like numpy, scikit-learn you get the idea). This used to be the system python install before externally managed installs came along. Then tools like pyenv were used to create a venv with this environment and put it on the $PATH using pyenv global venv_name. But not all tools (like uv) use this approach, they tend to go with the traditional approach of having venvs inside the project root (and I like this approach).

What does "global" mean to you in this instance?

So a virtual environment that is used as a fallback for when a project specific venv is not present. Like how the system install used to be used when a venv was not activated back in the days.

updating PATH

This is the first solution that comes to mind, why not put it on $PATH? Because then we go back to having the same problems externally managed installs solved. For instance: now we have an environment on $PATH that can be inadvertently modified and cause all types of headaches.

Plus, we get one more problem with this approach: venvs created using the python on $PATH will now be pointing to a venv! And if you have been following this, you'll know it's a hot mess of symlinks pointing to symlinks, etc, etc (mainly due to a lack of PEP for this IMO). And there are many other small degradations to user experience in daily use with this approach.

So what we need is a way to locate/use this global venv without putting it on $PATH or forced to use a tool that pet supports. Reading $VIRTUAL_ENV also has the potential benefit of detecting a venv that is activated but not locatable by pet maybe due to its "unconventional" location, apart from being tool-agnostic (for users who choose to set it manually).

[pantheraleo-7]

Also, I am exposed to pet via vscode and zed, which are editors. So whenever I open a data science script to have a quick look, I should not be expected to create a venv (each time, for a script) and then install those same packages repeatedly that I know I already have in a venv but the editor (actually pet) is not able to locate that even if I activate it.

If you have any other way, I'll be happy to hear.

[brettcannon]

whenever I open a data science script to have a quick look, I should not be expected to create a venv (each time, for a script) and then install those same packages repeatedly that I know I already have in a venv but the editor (actually pet) is not able to locate that even if I activate it.

Inline script metadata should eventually take care of that for you by creating and caching a virtual environment on the fly.