pixi environment not properly detected from 0.8.18

[atonderski]

Summary

Not sure if this is a bug or intentional but it definitely broke my workflow so I'm reporting it here.

$ pixi run uv pip install --upgrade --system uv==0.8.18
Using Python 3.12.0 environment at: .pixi/envs/default
Resolved 1 package in 69ms
Prepared 1 package in 586ms
Uninstalled 1 package in 0.15ms
Installed 1 package in 1ms
 - uv==0.8.12
 + uv==0.8.18
$ uv pip install tqdm
error: No virtual environment found; run `uv venv` to create an environment, or pass `--system` to install into a non-virtual environment

but when i downgrade, it works as before, and as I expect it should work

$ pixi run uv pip install --upgrade --system uv==0.8.17
Using Python 3.12.0 environment at: .pixi/envs/default
Resolved 1 package in 73ms
Prepared 1 package in 576ms
Uninstalled 1 package in 0.16ms
Installed 1 package in 1ms
 - uv==0.8.18
 + uv==0.8.17
$ uv pip install tqdm
Using Python 3.12.0 environment at: .pixi/envs/default
Resolved 1 package in 0.91ms
Installed 1 package in 2ms
 + tqdm==4.67.1

This is reproducible in a clean environment (using pixi 0.48.2)

pixi init
pixi add python=3.12
pixi add uv=0.8.18  # or any later, tested with 0.9.2 also
pixi run uv pip install tqdm  # will complain about env
pixi add uv=0.8.17
pixi run uv pip install tqdm  # will work

Platform

Linux 6.12.10-76061203-generic x86_64 GNU/Linux

Version

=0.8.18

Python version

Python 3.12.0

[zanieb]

The verbose logs are

❯ pixi init
✔ Created /private/tmp/example/pixi.toml
❯ pixi add python=3.12
✔ Added python=3.12
❯ pixi add uv=0.8.18
✔ Added uv=0.8.18
❯ pixi run uv pip install tqdm
error: No virtual environment found; run `uv venv` to create an environment, or pass `--system` to install into a non-virtual environment
❯ pixi run uv pip install tqdm -v
DEBUG uv 0.8.18
DEBUG Searching for default Python interpreter in virtual environments
DEBUG Found `cpython-3.12.12-macos-aarch64-none` at `/private/tmp/example/.pixi/envs/default/bin/python3` (conda prefix)
DEBUG Ignoring Python interpreter at `/private/tmp/example/.pixi/envs/default/bin/python3`: system interpreter not explicitly requested
DEBUG Searching for managed installations at `/Users/zb/.local/share/uv/python`
DEBUG Found managed installation `cpython-3.14.0-macos-aarch64-none`
DEBUG Ignoring Python interpreter at `/Users/zb/.local/share/uv/python/cpython-3.14.0-macos-aarch64-none/bin/python3.14`: system interpreter not explicitly requested
DEBUG Found managed installation `cpython-3.14.0+freethreaded-macos-aarch64-none`
DEBUG Ignoring Python interpreter at `/Users/zb/.local/share/uv/python/cpython-3.14.0+freethreaded-macos-aarch64-none/bin/python3.14t`: system interpreter not explicitly requested
DEBUG Found managed installation `cpython-3.14.0rc3-macos-aarch64-none`
DEBUG Ignoring Python interpreter at `/Users/zb/.local/share/uv/python/cpython-3.14.0rc3-macos-aarch64-none/bin/python3.14`: system interpreter not explicitly requested
DEBUG Found managed installation `cpython-3.13.8-macos-aarch64-none`
DEBUG Ignoring Python interpreter at `/Users/zb/.local/share/uv/python/cpython-3.13.8-macos-aarch64-none/bin/python3.13`: system interpreter not explicitly requested
DEBUG Found `cpython-3.12.12-macos-aarch64-none` at `/private/tmp/example/.pixi/envs/default/bin/python` (first executable in the search path)
DEBUG Ignoring Python interpreter at `/private/tmp/example/.pixi/envs/default/bin/python`: system interpreter not explicitly requested
DEBUG Ignoring Python interpreter at `/private/tmp/example/.pixi/envs/default/bin/python3`: system interpreter not explicitly requested
DEBUG Ignoring Python interpreter at `/private/tmp/example/.pixi/envs/default/bin/python3.12`: system interpreter not explicitly requested
DEBUG Ignoring Python interpreter at `/Users/zb/.local/bin/python3.14t`: system interpreter not explicitly requested
DEBUG Ignoring Python interpreter at `/Users/zb/.local/bin/python3.14`: system interpreter not explicitly requested
DEBUG Ignoring Python interpreter at `/opt/homebrew/bin/python3`: system interpreter not explicitly requested
DEBUG Ignoring Python interpreter at `/opt/homebrew/bin/python3.11`: system interpreter not explicitly requested
DEBUG Ignoring Python interpreter at `/opt/homebrew/bin/python3.12`: system interpreter not explicitly requested
DEBUG Ignoring Python interpreter at `/opt/homebrew/bin/python3.13`: system interpreter not explicitly requested
DEBUG Ignoring Python interpreter at `/usr/bin/python3`: system interpreter not explicitly requested
error: No virtual environment found; run `uv venv` to create an environment, or pass `--system` to install into a non-virtual environment

The key being

DEBUG Found `cpython-3.12.12-macos-aarch64-none` at `/private/tmp/example/.pixi/envs/default/bin/python3` (conda prefix)
DEBUG Ignoring Python interpreter at `/private/tmp/example/.pixi/envs/default/bin/python3`: system interpreter not explicitly requested

which is likely caused by #15679

[zanieb]

I think this is actually correct, in that you are writing the "default" environment which requires the --system flag — the logic for determining that the environment was the default was just wrong before.

[noamgot]

Just encountered this issue on my side, glad to see that someone else opened an issue :)
@zanieb so the verdict is to use the --system to install in the relevant pixi (conda) environment?
My use case is similar to this pixi example: https://github.com/prefix-dev/pixi/blob/main/examples/docker/pyproject.toml

(it's somewhat outdated, but the idea is that I want to install a locally built package in a production environment using up pip install)

[zanieb]

Yeah, I think so. If you create and activate a non-default environment with Pixi or Conda, that opt-in is no longer required.

[zanieb]

cc @tdejager

[tdejager]

I think that marking the interpreter found at:

DEBUG Found `cpython-3.12.12-macos-aarch64-none` at `/private/tmp/example/.pixi/envs/default/bin/python` (first executable in the search path)
DEBUG Ignoring Python interpreter at `/private/tmp/example/.pixi/envs/default/bin/python`: system interpreter not explicitly requested

As a system intepreter, could be incorrect.

To explain: like uv we install a version of python into what we call a prefix/environment, similar to a virtualenv. This environment is then activated on a pixi run, similar as to running an activation script in a virtual-env. Its just integrated into the command directly in pixi.

The default environment is basically the default that a user usually runs on, kind of like running uv venv default. So I wouldn't expect it to differ from any enviroment that can be created in pixi.

The question for me is that, whether or not passing in --system is correct depending on what is implied with a --system install. If this implies a global installation, then I think this a bug. If this implies a non-standalone version of python, then this may be correct. From first glance it seems incorrect though.

We can try fixing this in uv if this makes sense :)

[zanieb]

Do you feel that the default environment in Pixi differs from the base environment in Conda?

The intent is that you should not be able to install into unisolated environments, e.g., one that is not explicitly activated or selected, without the --system flag. The flag is sort of poorly named, but that's the concept underlying it.

There are some issues linked in #7691 too for more context.

[tdejager]

I feel that it does yeah, the default environment is per project and the base is across projects and thus your system. I think that would qualify the base environment as unisolated and the pixi environment as isolated. There is not a whole lot of difference to a virtual-env and a pixi-conda environment in that sense.

That said, looking at the linked issues, I feel that running uv pip list or whatever should actually prefer the uv activated venv over the base one. So maybe the logic needs to be rethought a bit?

[zanieb]

Oh, actually I didn't see that the pixi environment was scoped to the project. Sorry about that.

The logic is definitely wrong then.

❯ pixi run bash -c "env | grep CONDA"
CONDA_PREFIX=/private/tmp/example/.pixi/envs/default
CONDA_SHLVL=1
CONDA_DEFAULT_ENV=example

The problem is that your behavior here differs from Conda and breaks our heuristic. Specifically, the CONDA_PREFIX directory name differs from the CONDA_DEFAULT_ENV name.

I'm not quite sure how to repair that in a way that remains compatible with Conda. We could just special case the default path, but that's kind of weird.

You can see the whole heuristic at

uv/crates/uv-python/src/virtualenv.rs

Lines 80 to 127 in fa53a62

	/// Whether the given `CONDA_PREFIX` path is the base Conda environment.
	///
	/// The base environment is typically stored in a location matching the `_CONDA_ROOT` path.
	///
	/// Additionally, when the base environment is active, `CONDA_DEFAULT_ENV` will be set to a
	/// name, e.g., `base`, which does not match the `CONDA_PREFIX`, e.g., `/usr/local` instead of
	/// `/usr/local/conda/envs/<name>`. Note the name `CONDA_DEFAULT_ENV` is misleading, it's the
	/// active environment name, not a constant base environment name.
	fn from_prefix_path(path: &Path) -> Self {
	// If `_CONDA_ROOT` is set and matches `CONDA_PREFIX`, it's the base environment.
	if let Ok(conda_root) = env::var(EnvVars::CONDA_ROOT) {
	if path == Path::new(&conda_root) {
	return Self::Base;
	}
	}
	// Next, we'll use a heuristic based on `CONDA_DEFAULT_ENV`
	let Ok(current_env) = env::var(EnvVars::CONDA_DEFAULT_ENV) else {
	return Self::Child;
	};
	// If the `CONDA_PREFIX` equals the `CONDA_DEFAULT_ENV`, we're in an unnamed environment
	// which is typical for environments created with `conda create -p /path/to/env`.
	if path == Path::new(&current_env) {
	return Self::Child;
	}
	// If the environment name is "base" or "root", treat it as a base environment
	//
	// These are the expected names for the base environment; and is retained for backwards
	// compatibility, but in a future breaking release we should remove this special-casing.
	if current_env == "base" || current_env == "root" {
	return Self::Base;
	}
	// For other environment names, use the path-based logic
	let Some(name) = path.file_name() else {
	return Self::Child;
	};
	// If the environment is in a directory matching the name of the environment, it's not
	// usually a base environment.
	if name.to_str().is_some_and(|name| name == current_env) {
	Self::Child
	} else {
	Self::Base
	}
	}

[tdejager]

There should be $CONDA_PREFIX/conda-meta/pixi file that we can use to determine if its a pixi environment :)