Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[debian bookworm] Python Feature fails to install with 'error: externally-managed-environment' #577

Closed
joshspicer opened this issue Jun 16, 2023 · 14 comments · Fixed by #830

Comments

@joshspicer
Copy link
Member

ref: #576

Test run: https://github.com/devcontainers/features/actions/runs/5293497423/jobs/9581722477#step:4:1608

[2023-06-16T19:24:31.534Z] #18 36.34 Processing triggers for libc-bin (2.36-9) ...
#18 36.39 Installing Python tools...
#18 36.42 Updating pip...

[2023-06-16T19:24:31.810Z] #18 36.77 error: externally-managed-environment
#18 36.77 
#18 36.77 × This environment is externally managed
#18 36.77 ╰─> To install Python packages system-wide, try apt install
#18 36.77     python3-xyz, where xyz is the package you are trying to
#18 36.77     install.
#18 36.77     
#18 36.77     If you wish to install a non-Debian-packaged Python package,
#18 36.77     create a virtual environment using python3 -m venv path/to/venv.
#18 36.77     Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
#18 36.77     sure you have python3-full installed.
#18 36.77     
#18 36.77     If you wish to install a non-Debian packaged Python application,
#18 36.77     it may be easiest to use pipx install xyz, which will manage a
#18 36.77     virtual environment for you. Make sure you have pipx installed.
#18 36.77     
#18 36.77     See /usr/share/doc/python3.11/README.venv for more information.
[2023-06-16T19:24:31.810Z] 
#18 36.77 
#18 36.77 note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
#18 36.77 hint: See PEP 668 for the detailed specification.
@joshspicer
Copy link
Member Author

Looks like the azure-cli (which attempts to install python) is effected the same way: https://github.com/devcontainers/features/actions/runs/5294360726/jobs/9583631989

@vermegi
Copy link

vermegi commented Jun 23, 2023

Affirmative on azure-cli install being effected by this change on Debian bookworm as well: Azure/azure-cli#26737 and #589

Reverting to -bullseye, this is (for now) working fine.

vermegi added a commit to Azure-Samples/java-microservices-aks-lab that referenced this issue Jun 23, 2023
@jongio
Copy link

jongio commented Jun 26, 2023

Add "version": "latest"

"ghcr.io/devcontainers/features/azure-cli:1": {
			"version": "latest"
		}

@alfredodeza
Copy link

From what I can tell in the logs, specifically in this section:

[2023-06-16T19:23:54.049Z] #17 11.11 error: externally-managed-environment
#17 11.11 
#17 11.11 × This environment is externally managed
#17 11.11 ╰─> To install Python packages system-wide, try apt install
#17 11.11     python3-xyz, where xyz is the package you are trying to
#17 11.11     install.

That happens when installing a package directly with pip onto system-installed Python. It looks like there is a suggestion right afterwards:

You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages

This type of situation is usually avoided in development environments where developers will install libraries and packages in a virtualenv (which is also suggested).

I tried looking around the scripts that are running this test but couldn't figure out what exactly is doing these installs, but happy to offer some more help here if needed

@mering
Copy link

mering commented Sep 15, 2023

Azure/azure-cli#26737 seems to use pipx as a resolution

Other alternatives are listed here.

Another alternative would be to use the system packages if version == "os-installed".

@SebastianBalle
Copy link

SebastianBalle commented Oct 30, 2023

I have similar issues when I try to install docker-outside-docker. I get the error:

#13 32.44 error: externally-managed-environment
#13 32.44 
#13 32.44 × This environment is externally managed
#13 32.44 ╰─> To install Python packages system-wide, try apt install
#13 32.44     python3-xyz, where xyz is the package you are trying to
#13 32.44     install.
#13 32.44     
#13 32.44     If you wish to install a non-Debian-packaged Python package,
#13 32.44     create a virtual environment using python3 -m venv path/to/venv.
#13 32.44     Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
#13 32.44     sure you have python3-full installed.
#13 32.44     
#13 32.44     If you wish to install a non-Debian packaged Python application,
#13 32.44     it may be easiest to use pipx install xyz, which will manage a
#13 32.44     virtual environment for you. Make sure you have pipx installed.
#13 32.44     
#13 32.44     See /usr/share/doc/python3.11/README.venv for more information.

Even if I install python3-full before running the script I still get the same error. I use the following base image: mcr.microsoft.com/vscode/devcontainers/base:1.0.4-bookworm.

Update 20231031

Using the mcr.microsoft.com/vscode/devcontainers/base:1.0.4-bookworm with an arm64 architecture will fail in the installation unless the default value of docker compose is changed from v1 to v2.
In general the default version for docker compose should be v2 as v1 is no longer supported. The default value should either be modified by the maintainers or overwritten by the user.

@jtbandes
Copy link

jtbandes commented Nov 2, 2023

@SebastianBalle Do you know how to override this version to make the installation work on arm64? I tried editing devcontainer.json to look like this based on these docs:

  "features": {
    "ghcr.io/devcontainers/features/docker-in-docker:2": {
      "dockerDashComposeVersion": "v2"
    },

But that doesn't seem to work.

@SebastianBalle
Copy link

@jtbandes

We store the script hardcoded in our internal repository. We simply changed the default value as follow:

DOCKER_DASH_COMPOSE_VERSION=${7:-"v2"} # v1 or v2

In our devcontainer.json we have "dockerComposeFile": "docker-compose.yml", in which we have specified the Dockerfile that contains the above mentioned default value.

@samruddhikhandale samruddhikhandale self-assigned this Nov 7, 2023
@samruddhikhandale
Copy link
Member

samruddhikhandale commented Nov 7, 2023

In general the default version for docker compose should be v2 as v1 is no longer supported. The default value should either be modified by the maintainers or overwritten by the user.

Opened #745 for tracking v1 deprecation.

I should be able to get to #745 soon. Thanks for your patience.

@samruddhikhandale
Copy link
Member

As Docker v1 is deprecated, and Docker v2 works fine with debain:bookworm, we can simply close this issue once #745 is fixed.

Reasoning: As docker v1 is out of support by Docker, similarly, we can deprecate it's support and ask users to switch to v2.

@Clockwork-Muse
Copy link
Contributor

As Docker v1 is deprecated, and Docker v2 works fine with debain:bookworm, we can simply close this issue once #745 is fixed.

That's not sufficient. The python feature still fails to install on its own, regardless of whether or not any docker feature is installed. That is, the python feature is broken as well.

@Clockwork-Muse
Copy link
Contributor

Side note: Attempting to get a "partially working" feature install (mostly by disabling tool install) results in a broken setup. In particular, pipx is installed, but attempts to reference global locations when called, meaning it fails due to permissions issues. At the moment, installing python/pipx via the apt-packages feature is working for me.

@0xMihir
Copy link

0xMihir commented Jan 3, 2024

Yeah, I'm getting the same error with bookworm and v2:

{
	"name": "Rust",

	"image": "mcr.microsoft.com/devcontainers/rust:bookworm",
	"features": {
		"ghcr.io/devcontainers-contrib/features/apt-get-packages:2": {
			"packages": "software-properties-common, libzstd-dev"
		},
		"ghcr.io/devcontainers/features/docker-in-docker:2": {
			"dockerDashComposeVersion": "v2"
		}
	},
	"overrideFeatureInstallOrder": [
		"ghcr.io/devcontainers-contrib/features/apt-get-packages:2",
		"ghcr.io/devcontainers/features/docker-in-docker:2"
	],
	"appPort": "9042:9042",
	"remoteUser": "root",
	"onCreateCommand": "",
	"privileged": true,
	"customizations": {
		"vscode": {
			"extensions": [
				"1YiB.rust-bundle",
				"panicbit.cargo"
			]
		}
	},
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants