pip ignores the site.ENABLE_USER_SITE value if user is set explicitly (asymetric with venv behaviour)

[pelson]

Environment

• pip version: 20.2.0 & master
• Python version: all
• OS: all

Description

When we have a venv or virtualenv, and site.ENABLE_USER_SITE is False, and try to pip install --user and error is raised (InstallationError). If we do the same with a non-virtual environment, the ENABLE_USER_SITE value is ignored.

$ python -m venv /tmp/venv
$ source /tmp/venv/bin/activate
(venv) $ PYTHONNOUSERSITE=1 pip install --user typing-extensions
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.

But with a base environment (i.e. not virtual):

(venv) $ deactivate
$ PYTHONNOUSERSITE=1 pip -v install --user typing-extensions
Using pip 20.2.2 from /usr/local/lib/python3.8/site-packages/pip (python 3.8)
User install by explicit request
...
Collecting typing-extensions
  Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Installing collected packages: typing-extensions
Successfully installed typing-extensions-3.7.4.3

Expected behavior

I expect the site.ENABLE_USER_SITE to always be honoured

[pelson]

This came about because of a question on StackOverflow: https://superuser.com/questions/1509784/how-can-i-block-pip-and-python-user-packages. I'm interested in contributing a fix to this, and have submitted a PR in #8796.

[McSinyx]

According to the docs, pip should have the rights to install to user-site if ENABLE_USER_SITE is False and should only error when it is None (see this diff for what I mean). I don't know the answer to the linked thread on StackExchange though.

[pelson]

pip should have the rights to install to user-site if ENABLE_USER_SITE is False and should only error when it is None

Perhaps you got this the wrong way around? If it is False, user site packages are not enabled, and it wouldn't make sense to install packages into the user directory. The same holds for a value of None.

[uranusjr]

I think this depends on your interpretation to “disabled”. Does it mean that no Python tools can/should put stuffs inside the site, or simply that Python won’t load it into sys.path for import? pip’s current behaviour basically means that it’s installing into a location that’s neither discoverable nor importable (by default), i.e. implying the second interpretation.

IMO the linked Superuser question is fundamentally misguided. The only way to really prevent users from pip install-ing stuff into their user directories is to disable their filesystem write permission, since pip can always install to any arbitrary locations by specifying --prefix and --target. The --user option is not any different. The reason pip explicitly errors out inside a virtual environment is for UX reasons (a pip inside venv is not supposed to be able to modify the non-venv interpreter), and the same UX problem does not exist for site.ENABLE_USER_SITE = False since the installed packages are not visible anyway.

(To be explicit: I’m not saying your interpretation is wrong, just that pip’s behaviour is also not unreasonable.)

[pelson]

The reason pip explicitly errors out inside a virtual environment is for UX reasons (a pip inside venv is not supposed to be able to modify the non-venv interpreter)

Thanks for this extra context that I didn't have when reading the code originally, it is interesting and useful 👍.

The only way to really prevent users from pip install-ing stuff into their user directories is to disable their filesystem write permission

I appreciate the sentiment behind this statement, but I'm sure you can appreciate that this is rather coarse as a solution (even if it is the only sure-fire way of preventing installation to certain locations). In an organisation it doesn't necessarily follow that those who manage a Python installation also manage the user's filesystem. One already has the ability to prevent the user site being added to the path through documented Python mechanisms, and it would seem logical to me that such a configuration also prevents things being installed there (at least through the simple --user pip flag).

IMO the linked Superuser question is fundamentally misguided. Since pip can always install to any arbitrary locations by specifying --prefix and --target

Agreed on arbitrary install destinations. The question, as I interpret it, is not about "how to stop users installing stuff with pip to arbitrary locations", and more about "how to prevent novice users from getting themselves confused by installing to a special/global location". In that context I don't mind if users pip install --prefix=$(python -m site --user-site) the-thing, in precisely the same way that you can do to circumvent the limitations put in place for the UX you described for venvs.

This whole thing came up because I found it unusual to be able to disable user site-packages (from the sys.path), and yet still be able to install stuff into it with the --user flag. If there isn't any motivation to normalise what I would consider to be asymmetric behaviour, I'm completely OK with closing the issue down as a wontfix.

[uranusjr]

This whole thing came up because I found it unusual to be able to disable user site-packages (from the sys.path), and yet still be able to install stuff into it with the --user flag. If there isn't any motivation to normalise what I would consider to be asymmetric behaviour, I'm completely OK with closing the issue down as a wontfix.

Honestly I doubt the initial implementation actually put much thought in comparing approaches, and simply went with the most simplistic one (the current behaviour). I would welcome a PR containing a change of behaviour, with accompanying documentation explaining the thought process.

[pelson]

I would welcome a PR containing a change of behaviour, with accompanying documentation explaining the thought process.

I created #8796 to address the behaviour. That PR doesn't add the requested documentation nor suitable testing. I'd happily take that PR forwards if you could provide suggestions on where you think the documentation should live, and where you'd like to see the testing implemented for this feature.