Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable host callbacks and enforce 32bit precision when running on TPU #263

Merged
merged 3 commits into from
Oct 11, 2024

Conversation

flferretti
Copy link
Collaborator

@flferretti flferretti commented Oct 11, 2024

This PR introduces two main changes to allow runnning simulations on TPUs:

  1. host callbacks are disable by default, since they are not fully supported for TPU architectures
  2. 32bit precision is enforced when importing jaxsim

Example error on js.model.step:

[/usr/local/lib/python3.10/dist-packages/jax/_src/interpreters/pxla.py](https://localhost:8080/#) in __call__(self, *args)
   1275           or self.has_host_callbacks):
   1276         input_bufs = self._add_tokens_to_inputs(input_bufs)
-> 1277         results = self.xla_executable.execute_sharded(
   1278             input_bufs, with_tokens=True
   1279         )

XlaRuntimeError: INVALID_ARGUMENT: Unknown send channel 33 for host callbacks

📚 Documentation preview 📚: https://jaxsim--263.org.readthedocs.build//263/

@flferretti flferretti self-assigned this Oct 11, 2024
@flferretti flferretti force-pushed the feature/tpu_simulation branch 2 times, most recently from c0738a9 to 9dd5be1 Compare October 11, 2024 08:55
@flferretti flferretti force-pushed the feature/tpu_simulation branch 2 times, most recently from 6a3a423 to 9e44c6b Compare October 11, 2024 09:04
src/jaxsim/__init__.py Outdated Show resolved Hide resolved
Copy link
Member

@diegoferigo diegoferigo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @flferretti! Let me know what do you think of the suggestions below, there were cases that could be handled differently from what you proposed.

Please wait #262 to be merged before this one.

@diegoferigo diegoferigo force-pushed the feature/tpu_simulation branch from 515522f to a6080fc Compare October 11, 2024 09:36
@flferretti flferretti force-pushed the feature/tpu_simulation branch from a6080fc to 779d37b Compare October 11, 2024 10:19
@flferretti flferretti enabled auto-merge October 11, 2024 10:20
@flferretti flferretti merged commit 3ac2978 into main Oct 11, 2024
24 checks passed
@flferretti flferretti deleted the feature/tpu_simulation branch October 11, 2024 10:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants