[Feature] Return depth from RoboHiveEnv

[sriramsk1999]

Description

This PR adds a flag to enable returning of depth images from RoboHiveEnv.

Motivation and Context

Described in #2051
close #2051

• I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

What types of changes does your code introduce? Remove all that do not apply:

• New feature (non-breaking change which adds core functionality)
• Documentation (update in the documentation)

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

• I have read the CONTRIBUTION guide (required)
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

(I don't see any tests specific to RoboHiveEnv so I have not added any tests.)

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2058

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 15 New Failures, 1 Unrelated Failure

As of commit 9c39412 with merge base 6b87184 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Workflow failed! Resource not accessible by integration
• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Workflow failed! Resource not accessible by integration
• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 9a490ccbadca531289323496cf46c919d655ea7b515dd02e4fd2216d5ef88381 /exec failed with exit code 139
• Lint / c-source / linux-job (gh)
  RuntimeError: Command docker exec -t 2b2eaac8b9b00c3e3e9ccc798e4be565b33fa86c3f3f0386b3e6f902cb6a5883 /exec failed with exit code 1
• Lint / python-source-and-configs / linux-job (gh)
  RuntimeError: Command docker exec -t 71d086b8ffd0776e5d25dce92d4ac6a1743065e5ec79f929e7d7d1fb0a5eb209 /exec failed with exit code 1
• RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 40d52c1353e4198dc8c35790711d6b1b65f3982277819c14463e3bfe3c525993 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  test/test_transforms.py::TestVecNorm::test_parallelenv_vecnorm
• Unit-tests on Linux / tests-cpu (3.8) / linux-job (gh)
  test/test_transforms.py::TestVecNorm::test_parallelenv_vecnorm
• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh)
  test/test_transforms.py::TestVecNorm::test_parallelenv_vecnorm
• Unit-tests on Linux / tests-gpu (3.10, 12.1) / linux-job (gh)
  test/test_transforms.py::TestVecNorm::test_parallelenv_vecnorm
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  test/test_transforms.py::TestVecNorm::test_parallelenv_vecnorm
• Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh)
  test/test_transforms.py::TestVecNorm::test_parallelenv_vecnorm
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh)
  test/test_transforms.py::TestVecNorm::test_parallelenv_vecnorm
• Unit-tests on MacOS CPU / tests (3.8) / macos-job (gh)
  test/test_transforms.py::TestVecNorm::test_parallelenv_vecnorm
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  test/test_transforms.py::TestVecNorm::test_parallelenv_vecnorm

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Examples Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: This process waited for 1000.0 seconds without receiving a command from main. Consider increasing the maximum idle count if this is expected via the environment variable MAX_IDLE_COUNT (current value is 1000).

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[vmoens]

Cool! Can you also add a test in test_libs.py?

[sriramsk1999]

While adding the test, I noticed that the current test code was actually broken. I've raised PR #2062 with the changes, but some tests are failing. Will rebase and add new tests after that is merged.

[vmoens]

robohive tests raise an error bc the key can't be found, do you want me to investigate @sriramsk1999 ?

[sriramsk1999]

I got probably the same error, only for the rpFrankaRobotiqData environment.

AssertionError: The keys of the specs and data do not match:
    - List of keys present in real but not in fake: set(),
    - List of keys present in fake but not in real: {('next', 'depths'), 'depths'}.

I haven't gotten around to checking out the error in detail yet, please feel free to have a look :) Will update if I find a fix.

[sriramsk1999]

All RoboHive tests are now passing:

• rpFrankaRobotiqData-v0 is the only environment that has pre-registered visual_keys, so from_pixels is overridden to True even if the user sets it to False. I don't think this is expected behaviour, so I've removed this snippet in this commit
• I've added from_depths to the tests but there are now 2k tests just for RoboHive and it takes a while to complete (20 min on my PC)
• Unit tests were failing because the environment was already registered. I've enabled overriding the already registered env here

[vmoens]

LGTM