Pixi global inject

[smutch]

Problem description

When installing tools using pixi global install it is sometimes necessary to install additional packages in the same environment as the tool. For example pixi global install visidata will install visidata, but if we want to be able to inspect yaml files then we need pyyaml in the visidata environment too. With pipx, this is achieved using pipx inject visidata pyyaml. It would be great to have a similar command for pixi global.

Is there a current method to manually install a package inside a global tool environment? I tried manually activating the environment in ~/.pixi/envs/visidata but when I try using mamba install pyyaml I get the following error:

DirectoryNotACondaEnvironmentError: The target directory exists, but it is not a conda environment.

[baszalmstra]

This is a good idea!

[YeungOnion]

I'd be willing to work on this, would it be right to say if we want to have inject, then we essentially want pixi global to be a wrapper for project-type pixi?

That is, it'll find the project or init a project in ~/.pixi/envs as a bare pixi project repo (I mean bare as in the content of envs/my-package will be the contents of project level .pixi/ directory, env and prefix)

... then behave exactly as if using project-type pixi for that project? With some new mappings for subcommands like

• global install package => init --p $(uname -m), add package , setup ~/.pixi/bin with needed .sh scripts`
• global inject package depends => add depends, update project for package if needed

Does that make sense? I imagine once pixi global manages packages as if they were projects, it'll not diverge from the rest of pixi. The only question would be whether to follow pipx's approach of, don't global install things that don't have user entry points - I think it's a good approach. Thoughts?

[baszalmstra]

Mm, from my point of view global environments are indeed slimmed down pixi projects, but without a lot of the complexity of tasks, multiplatform, host/build dependencies and lock files. If you want that you're better of with a project. You can most likely reuse a lot of code but I would not quite treat them the same.

[dhirschfeld]

I think implementing global install as an ordinary pixi project with added magic of exposing the binaries/entry-points is a good concept that should allow reusing existing functionality.

Things I'd like:

• Allow adding pypi-dependencies and exposing entry-points from those packages as well
• Allow configuring which binaries / entry-points to expose - i.e. allow specifying entry-points that shouldn't be exposed.

[YeungOnion]

Another question on a use case is related to sharing the same tools. I haven't had the need to do this, but for example,

Someone in my collaboration asks me how I use some tool that I've installed with pipx (that has dependencies I've got via pipx inject), I'd give them code to run pipx from their shell.

Is this what we want for pixi? I think it helps keep the scope down since usage of globals are typically less context-sensitive,
BUT it's not as similar to project TOML's that only support [dev] dependencies and perhaps someone has a need for a more complex globally used tool. Writing it makes me realize I don't think I'd want that. Running from cli seems best I think.

[ruben-arts]

@YeungOnion If I understand your question correctly I think you're correct. We want to keep pixi global a tat more simple compared to a pixi project. We rather have you share pixi global install x y z as that is much more simple then sharing configuration files.

[YeungOnion]

Okay, thanks @ruben-arts, I think that's better too. @dhirschfeld maybe postponing exposing binaries from dependencies installed vua inject.

For now, scope is to only expose binaries that are already exposed with global install but adding an inject command for dependencies in those envs. I can do that.

[abkfenris]

I think implementing global install as an ordinary pixi project with added magic of exposing the binaries/entry-points is a good concept that should allow reusing existing functionality.

Things I'd like:

* Allow adding `pypi-dependencies` and exposing entry-points from those packages as well

* Allow configuring which binaries / entry-points to expose - i.e. allow specifying entry-points that shouldn't be exposed.

I was pondering this today and had the thought, why make it so that only global can expose binaries?

I'm taking some inspiration of the pixi run install task of developing pixi itself, but if ordinary pixi projects could expose entry-points globally, it would be really nice for developing tools locally in addition for installing downloaded ones.

pixi global install <package> then could become a wrapper around pixi init ~/.pixi/envs/<package> and something like pixi add --manifest-path ~/.pixi/envs/<package> --expose-bins <package>.

Then pixi global inject <parent_package> <new_package> would also wrap pixi add --manifest-path ~/.pixi/envs/<parent_package> and could have --expose-bins argument as well.

# ~/xpublish-dev/pixi.toml
[pypi-dependencies]
xpublish_cli = {path = "./xpublish-cli", editable = true, expose_bins = true}

Something like this would also help #1017 for where related subpackages also need to have their entry-points accessible.

# ~/.pixi/envs/ansible/pixi.toml
[dependencies]
ansible = {version = "something", expose_bins = true}
ansible-core = {version = "something", expose_bins = true}

[ruben-arts]

I feel that auto expose into your global space from a project wouldn't fit the pixi project vision. Possibly with a command line addition to allow a user to do it themselves could make sense.

[abkfenris]

I feel that auto expose into your global space from a project wouldn't fit the pixi project vision. Possibly with a command line addition to allow a user to do it themselves could make sense.

I was thinking along those lines, but I wanted to avoid the what-ifs in that comment to not confuse the idea too much.

A technique that I've seen to deal with this that I've liked is having a .local version of a config that acts as an override of the regular config file.

---

Say an pixi.toml has a dependency that exposes bins.

# ~/xpublish-dev/pixi.toml
[pypi-dependencies]
xpublish_cli = {path = "./xpublish-cli", editable = true, expose_bins = true}

Say, if you run pixi install (or any other commands that would trigger installation), then it would give a warning, but not expose them automatically.

WARN pixi::project: pixi.toml suggested exposing binaries and/or entry points from 
`xpublish_cli` to your global environment. 
Run `pixi install -expose-bins xpublish_cli` to add to your global environment.

If the user decides to run pixi install -expose-bins xpublish_cli, then they are linked in ~/.pixi/bin/, and a local config file is created (and added to .gitignore) to record the users choice.

# ~/xpublish-dev/pixi.local.toml
[pypi-dependencies]
xpublish_cli = {expose_bins = true}

To avoid nagging the user, there could be an --ignore-bins option as well that would also create a local config file.

# ~/xpublish-dev/pixi.local.toml
[pypi-dependencies]
xpublish_cli = {expose_bins = false}

[dhirschfeld]

Do binaries from a project need to be exposed globally?

Wouldn't pixi run <bibary-name> work just fine?

[dhirschfeld]

Just a data point - I tried to use copier as a pixi global app but it didn't work as I also need copier_templates_extensions installed in the same environment.

[tdejager]

Thanks @dhirschfeld good one! I think you might be able to do pixi exec -s copier -s copier_template_extensions copier, (haven't tried it 👼, so ymmv) to work around this for now.

[Hofer-Julian]

We are currently finishing up a minimal version of the new pixi global. The details are in this proposal, but the main advantages are:

• Shareable global environments.
• Managing complex environments with multiple packages as dependencies
• Flexible exposure of executables

Before we release it, we would appreciate it if a couple of users could try it out to make sure we are not missing glaring bugs or UX problems.
You can use it locally like this:

# Clone the repository
git clone https://github.com/prefix-dev/pixi.git

# Navigate into the repository
cd pixi

# Checkout a specific branch
git switch feature/pixi-global

# Install the Rust project with a custom binary name
pixi run install-as pixid

# Run commands
pixid global install nushell

This should work on all operating systems and only requires git and pixi to be installed locally.

[savente93]

Tried this on windows and got the following error:

 You don't have a global manifest yet.
Do you want to create one based on your existing installation?
 You don't have a global manifest yet.
Do you want to create one based on your existing installation?
Your existing installation will be removed if you decide against it. yes
  × Failed to create global manifest from existing installation
  ├─▶ failed to write to file `C:\Users\<username>\.pixi\manifests\pixi-
  │   global.toml`
  ╰─▶ The system cannot find the path specified. (os error 3)

The .pixi folder does exist, and creating the manifests folder in it before running the last command resolves the problem. I'll try it on my linux machine sometime today if I have the time

[Hofer-Julian]

Hey Sam 👋🏻 !

Nice find, I opened a PR that should fix this. Could you please check if it solves the problem?
#2163

[savente93]

I get a warning about add_channel being dead code, but otherwise works a treat :) not sure if I was supposed to test anything beyond that but the subcommand inject or global add isn't added yet to the version I tested. So I haven't tested that bit, but the installation works great. Not sure if it's supposed to but figured I'd mention it.

P.S. Hi Jullian :D

[savente93]

Also not sure if this is exactly the right place for it, but I just saw the open question in the proposal for "Should we version the manifest?" and I'm just gonna add my 2 cents unprompted: I highly recommend that you do. As someone that has worked a year on things that used to be un-versioned and had to make it versioned, please do your future selves a favor and do it, with a compatibility check if possible. Even a very lightweight versioning like a single number, can save you so much head ache in the future if you ever have to make changes to your format. Of course, it's still all of your decision, but from where I stand the potential benefits outweigh the cost

[savente93]

oh one other thing that is about this specific PR, the installation and running of tools via this version of pixi go very smoothly but after the command I run exits the terminal immediately closes. I tested this with several versions of hydromt (versions 0.10.0 and 0.9.4 specifically) as well as grayskull which was the first conda installable binary that came to mind.

Not sure if it is a pixi problem, but I don't remember it happening before, so it might be worth investigating on this side as well.

P.S. did some extra testing and this happens in multiple terminal emulators (wezterm which is my daily driver and the built in windows cmd, but not when running powershell) so it might not be pixi after all? I'm really not sure, just leaving it here in case, if I can do more to help diagnose this let me know

[Hofer-Julian]

Thanks for the feedback Sam ❤️