Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lock files / configuration for global environments #1280

Open
kxmh42 opened this issue Apr 26, 2024 · 5 comments
Open

Lock files / configuration for global environments #1280

kxmh42 opened this issue Apr 26, 2024 · 5 comments
Labels
✨ enhancement Feature request

Comments

@kxmh42
Copy link

kxmh42 commented Apr 26, 2024

Problem description

Pixi environments inside projects can be deleted at any time and regenerated using pixi.lock. However, it looks like it's not the case for environments created with pixi global install. If I delete $PIXI_HOME/envs, I have no way of recreating exact versions of the installed tools and their dependencies, is that right?

The particular scenario I'm looking at is saving space in my home folder by moving $PIXI_HOME/envs to a partition that has larger quota, but doesn't get backed up (see #997). And if $PIXI_HOME/envs is lost, I would like to recreate it in its exact state.

My suggestion would be to save lock files for global environments somewhere in $XDG_DATA_HOME/pixi / ~/.local/share/pixi.

@kxmh42 kxmh42 added the ✨ enhancement Feature request label Apr 26, 2024
@ruben-arts
Copy link
Contributor

Currently the global environments are simple "non project" installs of a conda environment. So we don't save any information apart form the environment itself.

I unfortunately can't find the discussion but we've been talking about a configuration file for the global environments. E.g.:

envs:
  - name: bat
    channels:
      - conda-forge
    dependencies:
      - bat

  - name: ripgrep
    channels:
      - conda-forge
    dependencies:
      - ripgrep

  - name: some_pytorch_package
    channels:
      - conda-forge
      - pytorch
    dependencies:
      - some_pytorch_package
      - pytorch==2.3.1

This way you could easily track your global environments and even better recreate them on a new machine. These might then be extended with lock files per env to accommodate your request.

p.s. Not sure if we want to do this in YAML but my brain wrote it like that. Just take this as an example.

@kxmh42
Copy link
Author

kxmh42 commented Apr 26, 2024

Your example configuration file is an equivalent of pixi.toml. Personally, I don't see a need for such a file. pixi global is a package manager, and package managers usually don't have such configuration files. What I think would be useful is an equivalent of pixi.lock, to keep track of versions of packages and their dependencies, so that they can be reinstalled with exactly the same versions.

@wolfv
Copy link
Member

wolfv commented Apr 26, 2024

@kxmh42 I think there are a few knobs users might want to turn in the "global" environemnts. For example, they might want to expose additional binaries or install additional (plugin) packages.

See #342 or #1017

Therefore we would likely come up with an additional config file that also does locking :)

@kxmh42
Copy link
Author

kxmh42 commented Apr 26, 2024

Ah, I see! In any case, if you do this, please don't save these config/lock files in $PIXI_HOME. Instead, put them in the appropriate XDG directories (config in $XDG_CONFIG_HOME/pixi, lockfile in $XDG_DATA_HOME/pixi, I guess?). This way, you can place $PIXI_HOME on a larger partition that is not backed up.

Edit: After reading the XDG spec, I believe the lockfile should be placed under $XDG_DATA_HOME, not $XDG_STATE_HOME.

@ruben-arts ruben-arts added this to the New `pixi global` milestone Aug 6, 2024
@ruben-arts ruben-arts changed the title Lock files for global environments Lock files / configuration for global environments Aug 6, 2024
@ruben-arts
Copy link
Contributor

An update for this Issue, We've now implemented a manifest in #1833

This will be the first step towards fixing this Issue. Would that already be enough for you?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✨ enhancement Feature request
Projects
None yet
Development

No branches or pull requests

3 participants