Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to install package globally for multi-user access #754

Closed
alextremblay opened this issue Oct 25, 2021 · 26 comments
Closed

Option to install package globally for multi-user access #754

alextremblay opened this issue Oct 25, 2021 · 26 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@alextremblay
Copy link
Contributor

Hi All, not sure if this qualifies as a feature request, or if it's already possible and just needs to be documented?

pipx by default installs its virtualenvs in a folder inside the user's home directory (ie ~/.local/pipx/venvs/), and installs the entrypoint binaries for those packages/virtualenvs in the user's personal path (ie ~/.local/bin).

Is it possible to install packages with pipx such that they are globally executable by all users on a system (ie install entrypoint binaries in /usr/local/bin, install virtualenvs somewhere that all users can access them?

How would this feature be useful?

Having this ability would make pipx a prime candidate for people looking to deploy python software in multi-user environments. Without this ability, pipx is not usable at all in such situations

Describe the solution you'd like

Ideally, some kind of --shared or --global flag would be the most straightforward solution, but even something like an optional environment variable would be nice

Ex: sudo pipx install --global cowsay

Describe alternatives you've considered

There are currently no alternatives / workarounds that I'm aware of. I cannot find anything about this in the documentation, the github issues, or on the web

@cs01
Copy link
Member

cs01 commented Oct 25, 2021

There are a couple environment variables that I think can do what you’re asking for.

See PIPX_HOME and PIPX_BIN_DIR.

https://pypa.github.io/pipx/docs/

@alextremblay
Copy link
Contributor Author

ahhhh, ok.

Apologies, i completely missed that in the documentation!

so if i understood correctly, a workaround to install globally accessible python apps with pipx on linux would be:

$ sudo PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin pipx install cowsay

is that right?

in that case, would pipx create /opt/pipx, or does it need to exist first?

@alextremblay
Copy link
Contributor Author

I just tested the above command, and it works exactly as intended!

didn't need to pre-create the directory or futz with directory permissions or anything :)

Is there any interest in me creating a PR to add this to the documentation?

@uranusjr
Copy link
Member

Yes, that would be awesome 🙂

@Jackenmen
Copy link
Contributor

It appears that this is already mentioned in the documentation:
https://pypa.github.io/pipx/installation/#installation-options

Is there any interest in exposing this in pipx for easier use or should users just be expected to create a shell alias if they want to use this regularly?

@Jackenmen
Copy link
Contributor

Something I observed while trying out this solution today - it is impossible to run pretty much all commands (e.g. pipx list) without being the owner of PIPX_HOME because pipx uses the logs subdirectory of PIPX_HOME for its logs and that directory isn't accessible to regular users when set to /opt/pipx. A workaround for this is giving access to the /opt/pipx/logs for everyone but that doesn't seem ideal.

@Jackenmen
Copy link
Contributor

Presumably, it would be safer to take additional pre-cautions such as only using the elevated permissions for installing the package, while building it with a normal user account. I imagine that this, however, might be out of scope for pipx, even if it would be incredibly cool if it could do that 😄

@uranusjr
Copy link
Member

We can probably detect the log directory’s permission, and log to somewhere the user is likely to own (e.g. use platformdirs) instead. This probably involves some tinkering with logging configuration, which is always tedious, so I’m probably not going to do that…

@Jackenmen
Copy link
Contributor

Jackenmen commented Jul 13, 2022

This probably involves some tinkering with logging configuration, which is always tedious, so I’m probably not going to do that…

What I hear is PRs welcome? :-) I might tinker with it later today.

@carlolars
Copy link

I think it would be nice to have a switch for global install. For example when using doas instead of sudo I had to first do doas su to run the pipx command with the environment variables as root. It would be a lot easier to just do doas pipx install --global cowsay.

@eode
Copy link

eode commented Jul 15, 2022

I definitely would like to have a --global option for pipx. I've put some thought into how this could be done (fairly) reasonably:

  • New environment vars:
    • PIPX_GLOBAL_HOME
    • PIPX_GLOBAL_BIN_DIR
  • Normal 'local' usage is treated just as it is now (including when using as root) except:
    • If the global directory is present, global info is included in output
    • --local can be specified to exclude global info
    • install output changes to "These apps are now locally available"
  • With --global:
    • Use PIPX_GLOBAL_* env vars.
      • Use PIPX_GLOBAL_HOME, or distro-specific global install dir, or /opt/pipx/ (failover)
      • Use PIPX_GLOBAL_BIN_DIR, the distro-specific global bin dir, or /usr/local/bin/ (failover)
    • Treat the global dir as though we have write permission -- user should have elevated permissions using sudo
  • Encourage proper usage via cli
    • Wherever a command looks like it might be mistaken, issue a warning / suggestion
      • "Writing into /home/someuser/.local/pipx, owned by someuser, as root. Did you mean to use --global?
      • "Installing foopkg locally for the user root. Did you mean to use --global?
  • JSON output includes "global venvs" key as a sibling to the "venvs" key.
  • Release on a major version change, because it introduces a new concept and changes the output.

Example usage:

$ pipx list
global venvs are in /opt/pipx/venvs
apps are exposed on your $PATH at /usr/local/bin
   package poetry 1.1.0, installed using Python 3.10.4
    - poetry
local venvs are in /home/someuser/.local/pipx/venvs
apps are exposed on your $PATH at /home/someuser/.local/bin
   package doit 0.36.0, installed using Python 3.10.4
    - doit
   package poetry 1.1.14, installed using Python 3.10.4
    - poetry
$ pipx list --local
local venvs are in /home/someuser/.local/pipx/venvs
apps are exposed on your $PATH at /home/someuser/.local/bin
   package doit 0.36.0, installed using Python 3.10.4
    - doit
   package poetry 1.1.14, installed using Python 3.10.4
    - poetry
$ sudo pipx install --global foocmd
  installed package foocmd 0.9.0, installed using Python 3.10.4
  These apps are now globally available
    - foo
    - foolog
done! ✨ 🌟 ✨
$ pipx install foocmd
  installed package foocmd 0.9.0, installed using Python 3.10.4
  These apps are now locally available
    - foo
    - foolog
done! ✨ 🌟 ✨

All-in-all, that's a fair amount of work, but not a ton of work to make the change, but I think that all of that work would be necessary to fully implement this change.

Alternately, --global could just do an override of the existing PIPX_HOME and PIPX_BIN_DIR env-vars, using distro-specific settings. This just means that sudo pipx list --global is required to read global items. I just like the UX of pipx list showing global and local installs.

@dukecat0 dukecat0 added the enhancement New feature or request label Aug 8, 2022
@silverwind
Copy link

silverwind commented Mar 10, 2023

This would be very useful to install binaries globally on a server for consumption by all users.

The workaround with PIPX_BIN_DIR sounds usable, but I would not want to alter PIPX_HOME as the assumption would be that pipx is already installed into a system directory (I think this is already the case when pipx is installed via apt).

@bulletmark
Copy link
Contributor

bulletmark commented Sep 5, 2023

I don't understand why this discussion assumes we need a --global option? Surely sudo pipx install cowsay should just install to the appropriate system directories? (It should keep private root managed pipx venvs in a system area of course). I notice that currently that command installs to /root/.local/bin/cowsay which I doubt anybody wants.

PS: To be clear, I guess I should point out that running code can trivially determine that is is running with root/sudo privilege so I am suggesting to exploit that.

@alextremblay
Copy link
Contributor Author

I don't understand why this discussion assumes we need a --global option? Surely sudo pipx install cowsay should just install to the appropriate system directories? (It should keep private root managed pipx venvs in a system area of course). I notice that currently that command installs to /root/.local/bin/cowsay which I doubt anybody wants.

PS: To be clear, I guess I should point out that running code can trivially determine that is is running with root/sudo privilege so I am suggesting to exploit that.

Though that sounds good in theory, I worry that it violates the principle of least surprise. Most package managers do not behave differently when run like that, and even pip, which operates globally by default, has an explicit ‘—user’ flag for when the default mode isn’t wanted

I would prefer to see an explicit ‘—global’ flag than implicit behaviour

@bulletmark
Copy link
Contributor

@alextremblay I was actually surprised it didn't work like this! I tried it and found it installed to /root/.local/bin/cowsay which seems non-sensical to me so I went looking and found this issue. I wouldn't use pip as a guide on how to do things as it is has all sorts of legacy obligations and I doubt anybody would design pip the way it is if we could start again.

@bulletmark
Copy link
Contributor

I created pipxx as a proof of concept and demonstration for what I am suggesting. It adds automatic root/global install as discussed here, and an enhanced list command output.

Just type pipx install git+https://github.com/bulletmark/pipxx to try it out.

@uranusjr
Copy link
Member

uranusjr commented Sep 6, 2023

Personally I also feel making sudo pipx install automatically install to system directories (instead of in /root/.local) is too implicit, for what it’s worth. A --system flag seems more in line. Maybe we should even block sudo pipx install to avoid the confusion (but still allow running pipx install as root, of course).

@bulletmark
Copy link
Contributor

Maybe we should even block sudo pipx install to avoid the confusion (but still allow running pipx install as root, of course)

Frankly, this statement horrifies me. That would break the honored orthogonal behavior of sudo and is inconsistent with what we expect and how we do things on Linux.

@uranusjr
Copy link
Member

uranusjr commented Sep 6, 2023

To clarify:

  • pipx install .. will work like it does now.
  • pipx install --system ... will not work unless you have write access to system locations (and if you do, installs to system locations).
  • sudo pipx install --system ... installs to system locations.
  • sudo pipx install ... will be ambiguous. We can either keep it working as-is (installs to /root/.local), but this can be surprising judging from the above discussion. So the idea is to maybe make this fail and suggest adding --system.

@bulletmark
Copy link
Contributor

@uranusjr after writing those rules out don't they seem convoluted, and certainly redundant? Compared to simply:

# Install for my user personally:
$ pipx install cowsay

# Or, install for all users globally:
$ sudo pipx install cowsay

pip has essentially been precluded as a global or user application installation tool nowadays (i.e. since the "externally-managed-environment" block most distros now apply to it) so pipx will step in to that role and thus global install capability needs to be improved and done asap. The pip UI is a mess so hopefully pipx designers will make better decisions.

@ketozhang
Copy link

ketozhang commented Sep 7, 2023

I disagree with sudo or --system as the UI.

Relying on sudo solely for the behavior has historically been bad UX. It's too implicits---my only intentions of using sudo pipx is to bypass permissions (often write permission as @uranusjr pointed out).

--system implies this mechanism is limited only to installing into conventional system directories (e.g., /usr/local/, /opt).


Taking a page off of NPM (analogously pipx was inspired by NPX), only one addition to @eode's suggestion:

  • --global behavior:
    • PIPX_HOME=${prefix:-/opt/}/pipx
    • PIPX_BIN_DIR=${prefix:-/usr/local}/bin
  • Default (or --local) behavior:
    • PIPX_HOME=${prefix:-~/.local}/pipx
    • PIPX_BIN_DIR=${prefix:-~/.local}/bin

these rules flexible using, for example, --prefix to set non-standard paths.

@haxwithaxe
Copy link

  1. I'm not advocating for this but given pipx is sort of an improved pip from a user perspective it would make sense to mimic the behavior of pip and install globally by default and fall back to installing per user.
  2. Is anyone actively working on this? If not I will add it to my short list of PRs to make.

@strk
Copy link

strk commented Dec 14, 2023

didn't need to pre-create the directory or futz with directory permissions or anything :)

It actually failed for me:

FileNotFoundError: [Errno 2] No such file or directory: '/home/opt/pipx/logs'

On the second run it worked, so some directories were created and some others weren't.
This was with pipx 1.0.0 as packaged in Ubuntu-22.04 (needs a sepearate ticket?)

@cyberfeign
Copy link

A bit late to the party since this seems to be natively implemented soon, but here is my quick function to add a --global option:

# Global pipx option
pipx() {
  if [[ "$@" =~ '--global' ]]; then
    args=()
    for arg in "$@"; do
      # Ignore bad argument
      [[ $arg != '--global' ]] && args+=("$arg")
    done
    command sudo PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin PIPX_MAN_DIR=/usr/local/share/man pipx "${args[@]}"
  else
    command pipx "$@"
  fi
}

Hope it works OK for someone else as well, not duly tested.

Gitznik pushed a commit that referenced this issue Mar 18, 2024
) (#1281)

* The bare minimum implementation of --global

* Added --global switch that sets the relevant paths to point to
  locations that should be accessable to all users.
* Added --global specific tests
* The implementation is crude.

* Fixing failing windows tests

...by not having --global in windows

The behavior intended by --global doesn't have a precedent in windows
when using pip (as in running pip with sudo). Leaving windows support
to another time after some discussion.

* Cleaned up the changes made to implement --global

* Moved path management to `pipx.paths`
* Path "constants" available via a context object `pipx.paths.ctx`

* Fixed some stuff missed in the merge

* Fix rebase issues

* Add missing mkdir calls, fix test_upgrade

* Add feature note to changelog.d

* Use decorator for test skips on windows

* Global tests update, PR cleanup

* Add note about --global switch in installation docs

* Revert unnecessary formatter changes

* Satisfy mypy import

* Add note about --global argument to how-pipx-works

* --global documentation improvements

* Update home_exists in make_global

* Run test_fetch_missing_python with setting local

* Add make_local at the end of all global tests

* Fix ensurepath

* make_local on fixture cleanup

* Revert "Fix ensurepath"

This reverts commit fe24c8f.

* Remove explicit make_local from tests

* Inform user about sudo pipx ensurepath --global

* Make test_cli test less tolerant

* Improve grammar

---------

Co-authored-by: haxwithaxe <spam@haxwithaxe.net>
@chrysle
Copy link
Contributor

chrysle commented Apr 21, 2024

This feature was added in #1281 and released as part of pipx 1.5.0!

@chrysle chrysle closed this as completed Apr 21, 2024
@eode
Copy link

eode commented May 3, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests