Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds image extracted features observation term and cartpole examples for it #1191

Merged

Conversation

glvov-bdai
Copy link
Collaborator

@glvov-bdai glvov-bdai commented Oct 8, 2024

Description

This adds an observation term to be able to easily extract features from the images, and adds a cartpole example of using this new term.

The new ResNet18 cartpole converges in less than 100 epochs.

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

I will update the version in the changelog and extension.toml after approval prior to merging in due to it causing merge conflicts when main updates

Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
glvov-bdai and others added 2 commits October 9, 2024 13:26
Signed-off-by: garylvov <67614381+garylvov@users.noreply.github.com>
@glvov-bdai glvov-bdai changed the title Add Image Extracted Features Observation and ResNet18 Cartpole Example Add Image Extracted Features Observation and Feature Extraction Cartpole Examples Oct 11, 2024
@Mayankm96 Mayankm96 changed the title Add Image Extracted Features Observation and Feature Extraction Cartpole Examples Add image extracted features observation term and cartpole examples for it Oct 12, 2024
@Mayankm96 Mayankm96 changed the title Add image extracted features observation term and cartpole examples for it Adds image extracted features observation term and cartpole examples for it Oct 12, 2024
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
@glvov-bdai glvov-bdai self-assigned this Oct 21, 2024
Copy link
Collaborator

@Dhoeller19 Dhoeller19 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm, thanks! Please update the changelog

Copy link
Collaborator

@jsmith-bdai jsmith-bdai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, thanks for adding! Just left some minor comments.

Just curious - how do you ensure the models are frozen? I didn't see it configured anywhere

glvov-bdai and others added 2 commits October 28, 2024 11:25
…vations.py

Co-authored-by: James Smith <142246516+jsmith-bdai@users.noreply.github.com>
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
@glvov-bdai
Copy link
Collaborator Author

glvov-bdai commented Oct 28, 2024

@jsmith-bdai Feature extraction encoders are frozen due to the .eval() flag in the model creation, like in

lambda: getattr(models, resnet_name)(pretrained=True).eval().to("cuda:0"),

Anyways, even if they weren't in eval mode I think they would still be frozen as I don't back-propogate the loss anywhere with model so weights shouldn't update

Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
@jsmith-bdai jsmith-bdai merged commit 9cc298e into isaac-sim:main Oct 28, 2024
4 of 5 checks passed
iamdrfly pushed a commit to iamdrfly/IsaacLab that referenced this pull request Nov 21, 2024
…for it (isaac-sim#1191)

# Description

This adds an observation term to be able to easily extract features from
the images, and adds a cartpole example of using this new term.

The new ResNet18 cartpole converges in less than 100 epochs.

## Type of change

<!-- As you go through the list, delete the ones that are not
applicable. -->

- New feature (non-breaking change which adds functionality)
- This change requires a documentation update

## Checklist
- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

I will update the version in the changelog and extension.toml after
approval prior to merging in due to it causing merge conflicts when main
updates

---------

Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
Signed-off-by: garylvov <67614381+garylvov@users.noreply.github.com>
Co-authored-by: garylvov <67614381+garylvov@users.noreply.github.com>
Co-authored-by: garylvov <gary.lvov@gmail.com>
Co-authored-by: David Hoeller <dhoeller@nvidia.com>
Co-authored-by: James Smith <142246516+jsmith-bdai@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants