Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replaces TensorDict with native dictionary #1348

Merged
merged 5 commits into from
Nov 4, 2024

Conversation

Toni-SM
Copy link
Contributor

@Toni-SM Toni-SM commented Oct 30, 2024

Description

Remove TensorDict usage from Isaac Lab

Type of change

  • Breaking change... for those ones using TensorDict specific API such as camera.data.output.to_dict() or camera.data.output.sorted_keys

Screenshots

Before

Screenshot from 2024-10-30 12-45-06

After

Screenshot from 2024-10-30 16-04-35

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@pascal-roth
Copy link
Collaborator

Before a more detailed review, I was wondering what is the reason why we should prefer normal dicts instead of Tensordict (thought we would rather use more and not less tensordicts in the code)?

@Toni-SM
Copy link
Contributor Author

Toni-SM commented Oct 31, 2024

Before a more detailed review, I was wondering what is the reason why we should prefer normal dicts instead of Tensordict (thought we would rather use more and not less tensordicts in the code)?

  1. Env API handles state, observation, action, reward, terminated, truncated and info.

    • For state, observation and action, we support fundamental and composite spaces (at least in Direct workflow): Box (tensor), Discrete (tensor), MultiDiscrete (tensor), Dict (dict), Tuple (tuple). TensorDict does not support tuples, so there is no point in using it for spaces in single or multi-agent envs.
    • For single-agent envs, reward, terminated and truncated are just tensors not so in multi-agent (dict of tensors)... but since it cannot be used for spaces, it does not make sense that some components use it and others do not.
  2. TensorDict is a PyTorch-scpecific thing which limits the implementation to that framework (making it difficult to scale Isaac Lab to other ML frameworks).

  3. Nested TensorDicts is pythonic ....
    381987801-9ccc351e-9771-40d0-80a5-f25444b687ba

@pascal-roth
Copy link
Collaborator

Okay makes sense to me, it was anyway basically the only place where we use them. @Mayankm96 if I remember correctly then you were in favor of using them, so tagging you here for viz

@Dhoeller19 Dhoeller19 changed the title Remove TensorDict usage Replaces TensorDict with native dictionary Nov 1, 2024
@kellyguo11 kellyguo11 merged commit c9e8631 into main Nov 4, 2024
4 of 5 checks passed
@kellyguo11 kellyguo11 deleted the toni/perf/remove_tensordict branch November 4, 2024 16:54
@vmoens
Copy link

vmoens commented Nov 13, 2024

Hey just a couple of comments based on the above discussion (not hoping that this PR will be reverted but I thought some of these comments were worth addressing)

Before a more detailed review, I was wondering what is the reason why we should prefer normal dicts instead of Tensordict (thought we would rather use more and not less tensordicts in the code)?

  1. Env API handles state, observation, action, reward, terminated, truncated and info.

    • For state, observation and action, we support fundamental and composite spaces (at least in Direct workflow): Box (tensor), Discrete (tensor), MultiDiscrete (tensor), Dict (dict), Tuple (tuple). TensorDict does not support tuples, so there is no point in using it for spaces in single or multi-agent envs.

TensorDict does support tuples (it supports basically any python object) although most APIs won't work (like to(device) will just ignore these values). If you need that we could call tree_map over non-tensors by default, that could actually make sense!

  1. TensorDict is a PyTorch-scpecific thing which limits the implementation to that framework (making it difficult to scale Isaac Lab to other ML frameworks).

On that I cannot disagree, it's definitely written for pytorch!

  1. Nested TensorDicts is pythonic ....
    381987801-9ccc351e-9771-40d0-80a5-f25444b687ba

hmmm ok?!

I'd just say that nested keys isn't really the main feature of the lib (and [k1][k2] or get(k1, {}).get(k2) will work perfectly too). The goal is to handle things like reshape ops / H2D transfers / arithmetic ops / functional calls / indexing etc as efficiently as possible across many tensors.

iamdrfly pushed a commit to iamdrfly/IsaacLab that referenced this pull request Nov 21, 2024
# Description

Remove TensorDict usage from Isaac Lab

## Type of change

- Breaking change... for those ones using TensorDict specific API such
as `camera.data.output.to_dict()` or `camera.data.output.sorted_keys`

## Screenshots

Before

![Screenshot from 2024-10-30
12-45-06](https://github.com/user-attachments/assets/2dc0e827-3e12-4ae9-849e-e9f75c718157)

After

![Screenshot from 2024-10-30
16-04-35](https://github.com/user-attachments/assets/715b6cb0-9f87-4938-8dbd-5c56203cb90e)

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it

For example,
- [x] I have done this task
- [ ] I have not done this task
-->

---------

Co-authored-by: Kelly Guo <kellyguo123@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants