Add pyproject.toml-based pixi support

[traversaro]

Just an experiment, as pixi 0.18.0 was released today with some convenient features:

• Just use pyproject.toml, no need for a separate pixi.toml file
• Support for editable install

Furthermore, the PR make extensive use of the multiple environments feature (see https://prefix.dev/blog/introducing_multi_env_pixi) to permit to separate run and test dependencies, and separate environments for gpu support (just on linux-64). In a nutshell the following 4 environments are available:

• default : the default environment, useful to run jaxsim on CPU
• test-cpu : environment with test dependencies, to test jaxsim on CPU
• gpu : environment with cuda dependencies, to run jaxsim on CPU or GPU
• test-gpu : environment with test and cuda dependencies, to test jaxsim on GPU

To check that this is actually working as intended, run import jax;print(jax.devices()) in the different environments:

traversaro@IITBMP014LW012:~/jaxsim$ pixi run -e default python -c "import jax;print(jax.devices())"
An NVIDIA GPU may be present on this machine, but a CUDA-enabled jaxlib is not installed. Falling back to cpu.
[CpuDevice(id=0)]
traversaro@IITBMP014LW012:~/jaxsim$ pixi run -e gpu python -c "import jax;print(jax.devices())"
[cuda(id=0)]

To run the tests, run:

pixi run test

If you are on machine with CUDA support, a menu will ask if you want to run test in test-cpu or test-gpu environment, otherwise the test will run automatically in test-cpu as a fallback.

No CI/docs for now as I first need to understand if it is actually useful.

To test this, make sure that you are using the latest pixi 0.18.0 .

I am not super excited by the size of the pixi.lock, but I guess it is the price to pay for full reproducibility and the use of multiple environments on multiple operating systems. Probably we can reduce this by removing some operating systems from the platform list.

---

📚 Documentation preview 📚: https://jaxsim--134.org.readthedocs.build//134/

[diegoferigo]

This is super cool @traversaro! I have some curiosity, left some comment.

I'm not super happy about the size of the lock file either, but let's see how it evolves. For the time being, do we have any use-case for arm machines? If we want to slim it down, maybe we can leave them out at least for now.

Last curiosity: how should the lock file be updated? Do we have to do it manually once in a while (or better, automate it every one/two weeks)?

[diegoferigo]

Should we specify here the minimum versions as setup.cfg?

[traversaro]

To be honest, I wanted to avoid having that information duplicated. We can probably just add a pip check somewhere before the tests to check if we are violating the setup.cfg constraints, and only add the constraint if necessary.

[diegoferigo]

That sounds good! I most cases, I guess that the solved environments are new enough.

[diegoferigo]

Suggested change

	[tool.pixi.feature.gpu]
	[tool.pixi.feature.gpu]

[traversaro]

It seems that there is no change?

[diegoferigo]

Nit, just blank line.

[traversaro]

The suggestion is not removing the blank file, probably there is a bug in GitHub's suggestion method.

[diegoferigo]

If we pin a minimum jaxlib version in tool.pixi.dependency, should we also copy it here?

[traversaro]

I guess so.

[diegoferigo]

Ok, I was wondering whether we have to duplicate it if a range is specified.

[traversaro]

Last curiosity: how should the lock file be updated? Do we have to do it manually once in a while (or better, automate it every one/two weeks)?

I added a CI to do it automatically in robotology/whole-body-estimators#178, see also the documentation in prefix-dev/pixi#909 . However, given the size of the pixi.lock I am not sure if such a frequent update is a good idea now.

[diegoferigo]

Last curiosity: how should the lock file be updated? Do we have to do it manually once in a while (or better, automate it every one/two weeks)?

I added a CI to do it automatically in robotology/whole-body-estimators#178, see also the documentation in prefix-dev/pixi#909 . However, given the size of the pixi.lock I am not sure if such a frequent update is a good idea now.

Thanks for the explanation! Given the youth of the tool, we can consider pixi support as experimental. Updating it manually when needed works for me.

[diegoferigo]

Wow the lock is about 1MB, pretty large file considering that the entire git repository after a couple of years of development is just 2.8 MB 😅

[traversaro]

Ok, I addressed the reviews. Leaving just linux-64 and depending on sdformat14 instead of gz-sim8 (as gz sdf is simply provided by sdformat14) I was able to reduce the pixi.lock size to 280KB, still not so small.

[traversaro]

Ok, I addressed the reviews. Leaving just linux-64 and depending on sdformat14 instead of gz-sim8 (as gz sdf is simply provided by sdformat14) I was able to reduce the pixi.lock size to 280KB, still not so small.

Having just a default and gpu environment instead of default,test-cpu,gpu,test-gpu the size is reduced to 230 Kb.

[diegoferigo]

Ok, I addressed the reviews. Leaving just linux-64 and depending on sdformat14 instead of gz-sim8 (as gz sdf is simply provided by sdformat14) I was able to reduce the pixi.lock size to 280KB, still not so small.

Perfect 😍

as gz sdf is simply provided by sdformat14

This is amazing! Since when? Last time I checked, gz was provided by Gazebo Sim and gz sdf was provided by a plugin from sdformat.

[traversaro]

as gz sdf is simply provided by sdformat14

This is amazing! Since when? Last time I checked, gz was provided by Gazebo Sim and gz sdf was provided by a plugin from sdformat.

Since the beginning I guess? gz/ign as always been provided by gz-tools/ign-tools (see https://github.com/gazebosim/gz-tools), and then all the packages install the plugins with their tools, including gz-sim that installs the gz sim command.

[diegoferigo]

as gz sdf is simply provided by sdformat14

This is amazing! Since when? Last time I checked, gz was provided by Gazebo Sim and gz sdf was provided by a plugin from sdformat.

Since the beginning I guess? gz/ign as always been provided by gz-tools/ign-tools (see https://github.com/gazebosim/gz-tools), and then all the packages install the plugins with their tools, including gz-sim that installs the gz sim command.

Ok, probably I overlooked things in the past. I always installed gz-sim* in my recent environments for nothing :) This also means that we can keep using gz sdf in ami-iit/rod#30.

[diegoferigo]

Is this PR ready for merging?

[traversaro]

Is this PR ready for merging?

Ready for me. Not super excited of the 200KB of space used, but I think the tradeoff with the added usability is worth it.

[tdejager]

Hopefully this will not explode the repo too much, because the file is diff-able so only changes should be recorded. However, it's still an initial add of 200KB of space for sure.

[diegoferigo]

Yep I agree, 200KB of a diff-able file is not too bad, especially if compared to the initial one 1MB large. Let's start playing with this real-case example of JaxSim, which is pretty interesting due to the need of GPU environments, and then re-evaluate it in a few months.

I'm particularly interested in the definition of a custom command that in a single line creates the CPU/GPU environment and then opens one of the existing jupyter notebooks with the examples. It would make the life of external users interested in playing with JaxSim much easier.

[traversaro]

I'm particularly interested in the definition of a custom command that in a single line creates the CPU/GPU environment and then opens one of the existing jupyter notebooks with the examples. It would make the life of external users interested in playing with JaxSim much easier.

We are almost there I guess, we need to add notebook as a dependency and add a task like https://github.com/traversaro/mujoco-mjx-pixi-example/blob/main/pixi.toml#L13 .