Add pixi support

[xela-95]

I would like to add a pixi project file to enable an easy way to install and run simulations in comodo in a way that is:

• reproducible across different machines
• easy to use
• that allows to only install the simulator one wants to use (mujoco, drake, JaxSim)

Blocked by:

• Add support for jaxsim simulator with examples #20
• Refactor contact models and add tests jaxsim#260

[xela-95]

How to use pixi in comodo

The pixi.toml file, also called project manifest, is where we specify all the project configurations, environments along with their dependencies.

In this case, I've created a default environment containing all the dependencies Comodo needs (taken from its README). Then I specified an environment for each simulator supported today on comodo: mujoco, JaxSim and Drake (in future we can add all the other simulators comodo will support in an extensible way).

These environment have the name of the simulator (i.e. they are called mujoco, jaxsim, drake), and there is also an all environment that include all of them. These environments build upon the default one and add just the dependencies they need to work properly.

The it is possible to activate one of the environments with:

pixi shell -e <environment-name>

just like one will activate a conda environment.

It is also possible to run directly a command in one of these environment using:

pixi run -e <environment-name> python <script-filename.py>

or

pixi run -e <environment-name> jupyter notebook

and then run the jupyter notebooks in the examples folder.

Visual Studio Code

For the folks using vscode, remember to select the appropriate pixi environment when developing and running python scripts by:

• Clicking on the python environment button on the bottom right corner of the IDE:
  
• and selecting the pixi environment from the dropdown list (among the available ones):
  

And when running Jupyter notebooks from vscode, it is possible to select as kernel one of the pixi environments bu clicking on the top right corner of the notebook:

[xela-95]

Issue installing Drake package

I'm getting an error when trying to add drake package, available from pypi as:

 pixi add -f drake --pypi drake                                                                                                                     
  × failed to solve the pypi requirements of 'all' 'linux-64'
  ├─▶ failed to resolve pypi dependencies
  ╰─▶ Because only drake>=0.35.0 is available and drake>=0.35.0,<=1.11.0 has no wheels with a matching Python ABI tag, we can conclude that drake<0.36.0 cannot be used.
      And because drake>=1.12.0 has no wheels with a matching Python implementation tag and you require drake, we can conclude that your requirements are unsatisfiable.
      
      hint: Pre-releases are available for drake in the requested range (e.g., 0.35.0b1), but pre-releases weren't enabled (try: `--prerelease=allow`)

[traversaro]

Issue installing Drake package

I'm getting an error when trying to add drake package, available from pypi as:

 pixi add -f drake --pypi drake                                                                                                                     
  × failed to solve the pypi requirements of 'all' 'linux-64'
  ├─▶ failed to resolve pypi dependencies
  ╰─▶ Because only drake>=0.35.0 is available and drake>=0.35.0,<=1.11.0 has no wheels with a matching Python ABI tag, we can conclude that drake<0.36.0 cannot be used.
      And because drake>=1.12.0 has no wheels with a matching Python implementation tag and you require drake, we can conclude that your requirements are unsatisfiable.
      
      hint: Pre-releases are available for drake in the requested range (e.g., 0.35.0b1), but pre-releases weren't enabled (try: `--prerelease=allow`)

So, how pixi solve works is that it first solves the conda dependencies, then given the installed version of conda dependencies it solves the python one. Which version of python are you using (or is pixi selecting?) I guess there is no drake version compatible with the required python version.

[traversaro]

This is manifesting in our setup for the jaxsim package.

Why? If we are installing jaxsim as a conda package, I don't think uv may complain about the git lfs files, or I am missing something? What error do you get if you try to install jaxsim without GIT_LFS_SKIP_SMUDGE ?

[xela-95]

So, how pixi solve works is that it first solves the conda dependencies, then given the installed version of conda dependencies it solves the python one. Which version of python are you using (or is pixi selecting?) I guess there is no drake version compatible with the required python version.

In my case pixi is selecting python version 3.10.15. Checking on the drake pypi page it says it supports python >= 3.10.

[xela-95]

Why? If we are installing jaxsim as a conda package, I don't think uv may complain about the git lfs files, or I am missing something? What error do you get if you try to install jaxsim without GIT_LFS_SKIP_SMUDGE ?

You're right @traversaro , my message was a regression when I was installing it from git source. Installing the onda package works fine. I tried multiple times, by cleaning the cache and existing environments and it worked fine. I will mark my message as obsolete.

[xela-95]

So, how pixi solve works is that it first solves the conda dependencies, then given the installed version of conda dependencies it solves the python one. Which version of python are you using (or is pixi selecting?) I guess there is no drake version compatible with the required python version.

Looking in Drake documentation, from the "Supported configurations" table reported here, it shows that for Ubuntu 24 LTS is supported only python 3.12, but the footnote says that in theory also python 3.10 should be supported via pip.

[xela-95]

BTW when I install this environment I get the following warning:

 WARN The package `etils==1.9.4` does not have an extra named `epy`

@traversaro I've seen you're one of the maintainers of the etils feedstock. Have you ever encountered this?

From a quick look at the etils package I've seen that there exist and epy extra: https://github.com/google/etils/blob/08f2c4767761240550d7040c7a9f11a7d1d83e75/pyproject.toml#L76-L78, but maybe is not included in the conda package recipe?

[traversaro]

BTW when I install this environment I get the following warning:

 WARN The package `etils==1.9.4` does not have an extra named `epy`

@traversaro I've seen you're one of the maintainers of the etils feedstock. Have you ever encountered this?

From a quick look at the etils package I've seen that there exist and epy extra: https://github.com/google/etils/blob/08f2c4767761240550d7040c7a9f11a7d1d83e75/pyproject.toml#L76-L78, but maybe is not included in the conda package recipe?

I am unable to reproduce the warning, what command are you executing?

[traversaro]

Issue installing Drake package

I'm getting an error when trying to add drake package, available from pypi as:

 pixi add -f drake --pypi drake                                                                                                                     
  × failed to solve the pypi requirements of 'all' 'linux-64'
  ├─▶ failed to resolve pypi dependencies
  ╰─▶ Because only drake>=0.35.0 is available and drake>=0.35.0,<=1.11.0 has no wheels with a matching Python ABI tag, we can conclude that drake<0.36.0 cannot be used.
      And because drake>=1.12.0 has no wheels with a matching Python implementation tag and you require drake, we can conclude that your requirements are unsatisfiable.
      
      hint: Pre-releases are available for drake in the requested range (e.g., 0.35.0b1), but pre-releases weren't enabled (try: `--prerelease=allow`)

Can you add python = "*" among the dependencies ? Doing that solves the problem for me:

index c078c78..a98a258 100644
--- a/pixi.toml
+++ b/pixi.toml
@@ -23,6 +23,7 @@ mesalib = ">=24.2.4,<25"
 notebook = ">=7.2.2,<8"
 ipykernel = ">=6.29.5,<7"
 resolve-robotics-uri-py = ">=0.3.0,<0.4"
+python = "*"

 [pypi-dependencies]
 # TODO: move adam-robotics package to conda dependencies as soon as PR https://github.com/ami-iit/adam/pull/100 is merged

However, the confusing error is indeed a pixi problem, if you do it in a simple case you get the clear error that a python is missing from dependencies.

[xela-95]

Can you add python = "*" among the dependencies ? Doing that solves the problem for me:

index c078c78..a98a258 100644
--- a/pixi.toml
+++ b/pixi.toml
@@ -23,6 +23,7 @@ mesalib = ">=24.2.4,<25"
 notebook = ">=7.2.2,<8"
 ipykernel = ">=6.29.5,<7"
 resolve-robotics-uri-py = ">=0.3.0,<0.4"
+python = "*"

 [pypi-dependencies]
 # TODO: move adam-robotics package to conda dependencies as soon as PR https://github.com/ami-iit/adam/pull/100 is merged

However, the confusing error is indeed a pixi problem, if you do it in a simple case you get the clear error that a python is missing from dependencies.

Strange, even doing this I continue to get the same error (I also removed the pixi.lock and cleaned previous environments).

I tried in an empty project to do:

pixi init

pixi add drake --pypi

and got: × missing python interpreter from environment help: Use pixi add python to install the latest python interpreter.

But doing

pixi add python
pixi add drake --pypi

Then gives the same:

  × failed to solve the pypi requirements of 'default' 'linux-64'
  ├─▶ failed to resolve pypi dependencies
  ╰─▶ Because only drake>=0.35.0 is available and drake>=0.35.0 has no wheels with a matching Python ABI tag, we can conclude that drake<0.36.0 cannot be used.
      And because you require drake, we can conclude that your requirements are unsatisfiable.
      
      hint: Pre-releases are available for drake in the requested range (e.g., 0.35.0b1), but pre-releases weren't enabled (try: `--prerelease=allow`)

[xela-95]

I am unable to reproduce the warning, what command are you executing?

I remember it was logged with pretty much every pixi command on this environment, but I also cannot reproduce this now, strange.. I will let you know when if this happens again.

[xela-95]

I am unable to reproduce the warning, what command are you executing?

I remember it was logged with pretty much every pixi command on this environment, but I also cannot reproduce this now, strange.. I will let you know when if this happens again.

Today, running pixi run test in for the Jaxsim repo I obtained the warnings:

$ pixi run test                                                                                                                                              ✔  12:01:40  
 WARN The feature 'style' is defined but not used in any environment
 WARN The feature 'testing' is defined but not used in any environment
 WARN The feature 'viz' is defined but not used in any environment
 WARN The feature 'all' is defined but not used in any environment
 WARN The package `etils==1.9.4` does not have an extra named `epy`
 WARN The package `etils==1.9.4` does not have an extra named `epy`
 WARN The package `etils==1.9.4` does not have an extra named `epy`
 WARN The package `etils==1.9.4` does not have an extra named `epy`

[xela-95]

@CarlottaSartore I think this PR is ready for review. To install pixi you can follow https://pixi.sh/latest/ and to use the pixi environment(s) of comodo you can follow the instructions in #23 (comment). Unfortunately is still not possible to install Drake automatically through pixi, we can address this in a future PR.

Let me know if that works for you :) we can also add instructions on the README if you want

[traversaro]

Unfortunately is still not possible to install Drake automatically through pixi, we can address this in a future PR.

I reported the issue in prefix-dev/pixi#771 . I think that Ruben is as ROSCon so I do not think he will able to reply shortly, but at least we reported it somewhere.

[xela-95]

I reported the issue in prefix-dev/pixi#771 . I think that Ruben is as ROSCon so I do not think he will able to reply shortly, but at least we reported it somewhere.

Thanks a lot @traversaro !!

[xela-95]

Updated adam-robotics package since ami-iit/adam#100 has been merged.

[xela-95]

With the suggestion from prefix-dev/pixi#771 (comment) I was able to install drake through pixi.

Now the drake example notebook runs, but still I'm getting an unrelated error while running the simulation:

[ERROR] [2024-10-30 11:20:36.872] [thread: 548825] [blf] [QPTSID::advance] osqp was not able to find a feasible solution.
[ERROR] [2024-10-30 11:20:38.349] [thread: 548825] [blf] [QPTSID::advance] osqp was not able to find a feasible solution.
ERROR:root:MultibodyPlant's discrete update solver failed to converge at simulation time = 4.391 with discrete update period = 0.001. This usually means that the plant's discrete update period is too large to resolve the system's dynamics for the given simulation conditions. This is often the case during abrupt collisions or during complex and fast changing contact configurations. Another common cause is the use of high gains in the simulation of closed loop systems. These might cause numerical instabilities given our discrete solver uses an explicit treatment of actuation inputs. Possible solutions include:
  1. reduce the discrete update period set at construction,
  2. decrease the high gains in your controller whenever possible,
  3. switch to a continuous model (discrete update period is zero),      though this might affect the simulation run time.

I don't know if that notebook worked successfully before @CarlottaSartore

[traversaro]

It may be a problem of drake version? @akhilsathuluri may know more about this. By checking on his lock files, I guess he used drake 1.23.0 .

[xela-95]

It may be a problem of drake version? @akhilsathuluri may know more about this. By checking on his lock files, I guess he used drake 1.23.0 .

I tried just by pinning that version but still I get the failed controller error, I have to try to run it with the same conda environment he used to check if it's a problem with dependencies or if the code the notebook uses has changed, breaking its functionality.

[traversaro]

If everything else is working except for drake perhaps we can merge as it is so people that anyhow use drake for the rest of the backends, and open a different issue to track solving the drake problems? fyi @xela-95 @CarlottaSartore

[xela-95]

If everything else is working except for drake perhaps we can merge as it is so people that anyhow use drake for the rest of the backends, and open a different issue to track solving the drake problems? fyi @xela-95 @CarlottaSartore

Yep good idea!

[xela-95]

Opened #27 since I'm getting the same error also from the main branch with a conda environment.

[xela-95]

@CarlottaSartore this is ready to be reviewed :)

[CarlottaSartore]

While testing I got the following error

but was due to my setup

[CarlottaSartore]

LGTM @xela-95 thanks! could you please add some documentation also in the main readme ?

[xela-95]

LGTM @xela-95 thanks! could you please add some documentation also in the main readme ?

Sure! I will add it ASAP

[xela-95]

While testing I got the following error

but was due to my setup

This is an issue with file descriptors, it happened seldomly also on my setup. I circumvented the problem by temporarily increasing the number of file descriptors in the terminal by executing: ulimit -n 65535. The reason why the file descriptors are exhausted is unknown to me, maybe is due to the high parallel number of download requests during the first installation of the environment.

[traversaro]

Interestingly, it would be worth reporting this problems and its workaround somehow, for example in https://github.com/prefix-dev/pixi/discussions (as I do not know if this is a issue that can be addressed).

[xela-95]

LGTM @xela-95 thanks! could you please add some documentation also in the main readme ?

@CarlottaSartore I refactored the README to be more clear and ordered. Let me know what do you think :)

[CarlottaSartore]

Thank you really much @xela-95 , few comments

[CarlottaSartore]

perfect ! Thanks @xela-95 ! Merging !