Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mismatching virtual package error should hint to solution (set system-requirements) #346

Open
pavelzw opened this issue Sep 19, 2023 · 12 comments · Fixed by #348
Open

Mismatching virtual package error should hint to solution (set system-requirements) #346

pavelzw opened this issue Sep 19, 2023 · 12 comments · Fixed by #348
Assignees
Labels
needs-decision Undecided if this should be done ✨ enhancement Feature request 👋 good first issue Good for newcomers

Comments

@pavelzw
Copy link
Contributor

pavelzw commented Sep 19, 2023

Problem description

$ pixi init
$ pixi add python
$ pixi install
x The current system has a mismatching virtual package. The project requires '__linux' to be at least version '5.10' but the system has version '4.18.0'

should be something like

$ pixi init
$ pixi add python
$ pixi install
x The current system has a mismatching virtual package. The project requires '__linux' to be at least version '5.10' but the system has version '4.18.0'. Try setting the following in your pixi.toml:
[system-requirements]
linux="4.18"

Similar for other virtual packages.

@pavelzw pavelzw added the ✨ enhancement Feature request label Sep 19, 2023
@baszalmstra baszalmstra added the 👋 good first issue Good for newcomers label Sep 19, 2023
baszalmstra pushed a commit that referenced this issue Sep 20, 2023
Add tip to the error of a to low local system requirement.

closes #346
@jpursell
Copy link

jpursell commented May 3, 2024

This should be re-opened. I just got this unhelpful error

bash-4.4$ pixi shell
  × The current platform does not satisfy the minimal virtual package requirements
  ╰─▶ the project does not support 'linux-64'

pixi version 0.20.1

@pavelzw
Copy link
Contributor Author

pavelzw commented May 3, 2024

@ruben-arts can you reopen please? I don't have permissions to do so

@ruben-arts ruben-arts reopened this May 4, 2024
@jpursell
Copy link

jpursell commented May 4, 2024

I should mention that the fix for this was not to add something to [system-requirements], but rather I needed to add "linux-64" to the platforms. It be very helpful if pixi would tell the user that.

@roaldarbol
Copy link

roaldarbol commented May 4, 2024

I also just got this for a pypi install:

pixi add --pypi syncstart==1.1.1
  × failed to solve the pypi requirements of 'default' 'osx-64'
  ├─▶ Failed to build: `opencv-python==4.9.0.80`
  ├─▶ Failed to install requirements from build-system.requires (install)
  ├─▶ Failed to download and build distributions
  ├─▶ Failed to fetch wheel: numpy==1.22.2
  ├─▶ Failed to build: `numpy==1.22.2`
  ╰─▶ Build backend failed to build wheel through `build_wheel()` with exit status: 1

... along with the longer stdout stuff.
It was resolved by setting:

[system-requirements]
macos = "11.0"

@ruben-arts
Copy link
Contributor

ruben-arts commented May 5, 2024

Honestly the users should feel the power of system requirements and not the downside. I've got a proposal to solve this issue with some workflow improvements:

TL;DR: Make pixi more flexible and let it interact with the dependency resolution automatically, and help the user by doing the work for them.

Definitions

Virtual packages: a dependency of a package that can not be provided as a conda package, these are system specifications or libc implementation on the system. e.g. cuda, linux, macos, glibc etc.
System requirements: Pixi's take on defining a virtual package in the project's manifest. It defines the requirements the project has of the machine it needs to run on, which it can't provide itself. There is a set of defaults per platform, these are used during the solve of the dependencies to avoid the solve to only work on your specific machine. If the project requires a different set then the default they need to be defined in the [system-requirements] table.

Solution

Let's make pixi solve with the current available virtual packages. Then after the solve figure out what virtual packages are required. If those differ from the default virtual packages they will automatically be added to the manifest file as system requirements. This goes both ways, having higher or lower versions on your current machine.

Solution to newly created problem

The down side of change would be that the solution(lock file) might only work for the latest machines as pixi picks the latest versions available for the given virtual packages. Potentially lowering the amount of machines your project can run on. This could be fixed by giving pixi an interactive option to re-solve for your machine, skipping the system-requirements given by the project. When it finds a different solution for your machine it automatically writes it back to the manifest, allowing the users to track these changes in vcs.

Example of the interactive prompt:

> pixi install
The project requires your system to have `cuda 12.0`. 
Should I try to solve a new environment with this machine's specs?
 > Yes
 - No

Edit:
After some discussion about the solution to this issue. Here is a more specific resulting situation:

$ pixi info
...
  Virtual packages: __cuda=12.5=0
...
$ pixi add pytorch
[system-requirements]
# Add system requirements that are non default
cuda = "12.1"

Change I expect to be needed:

  • Pixi needs to share the machines virtual packages instead of the projects requirements with the solver.
  • Pixi needs to figure out the version of the virtual packages from the locked environment.
    • For conda packages it easy to just look add the dependencies of all the repodata records
    • For pypi the information can be read from the wheel names. Still have to figure out which wheel to select but the first version should have the same behavior as of right now.

ping @baszalmstra @wolfv for input

@ruben-arts ruben-arts added the needs-decision Undecided if this should be done label May 5, 2024
@roaldarbol
Copy link

roaldarbol commented May 7, 2024

I think it IS really powerful, so I hope it will become super intuitive! For lots of tracking software (e.g. https://github.com/DeepLabCut/DeepLabCut, https://gitlab.com/polavieja_lab/idtrackerai) installation has been a major sticking point for adoption, especially since they depend on TensorFlow and thus getting cuda just right across platforms - this would be greatly improved by a simple specification.

For me personally, I think one of the confusing aspects have been around the OS-named system-requirements, and how they differ from platforms. E.g. if I have already specified osx-64 as the platform, it's been unintuitive why I'd need to additionally specify macos = "11.0" under system-requirements. Usually, the version requirements is to narrow the solving, so it took me by surprise that you could get something to solve by specifying a version - I'd think it would otherwise just attempt to solve against all system requirements for the specified platform.

Lastly, it would be great if pixi could "figure out" the correct requirement for the user's current platform. E.g. for cuda, It would make life easier for users if they didn't have to know the CUDA requirements for their particular platform. Here the target audience are not super tech-savvy, but would like to use techy tools easily. I guess that's part of @ruben-arts's proposal - is that correctly understood?

@baszalmstra
Copy link
Contributor

I think your idea can work @ruben-arts but I cant really see the ramifications yet. Lets try to build an experiment in the near future.

@ruben-arts
Copy link
Contributor

Hey @roaldarbol, Thanks for the input, this is good to take into account in the UX design. I believe that the things you mention are all atleast what I tried to explain in my proposal. So lets see what we can come up with in the first version!

@zachcp
Copy link

zachcp commented May 14, 2024

I just ran into this as well and wasn't sure what it meant.

$ pixi run test
Pixi task (test in default): pytest tests
  × The current system has a mismatching virtual package. The project requires
  │ '__linux' to be at least version '5.10' but the system has version '5.4.0'

@ruben-arts
Copy link
Contributor

@zachcp Yep exact same issue.

Try:

# [tool.pixi.system-requirements] for pyproject.toml
[system-requirements]
linux = "5.4"

@zoobab
Copy link

zoobab commented Dec 6, 2024

I have this issue on a centos 7 distro with an old kernel Linux version 3.10.0-1160.123.1.el7.x86_64:

$ pixi run start
  × The current system has a mismatching virtual package. The project requires '__linux' to be at least
  │ version '4.18' but the system has version '3.10.0'

@noamgot
Copy link

noamgot commented Dec 24, 2024

Encountered this one today (in a gitlab-ci pipeline):

$ pixi run py-check
Pixi task (py-check in lint): ruff check; ruff format --check
  × The current system has a mismatching virtual package. The project requires
  │ '__linux' to be at least version '4.18' but the system has version
  │ '4.15.0'

It failed despite setting in pixi.toml:

[system-requirements]
linux = "4.15"

Since my project also contains pyproject.toml file, I also tried to remove the former change and write in pyprpoject.toml:

[tool.pixi.system-requirements]
linux = "4.15"

both failed with the error mentioned above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-decision Undecided if this should be done ✨ enhancement Feature request 👋 good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants