Episode Creator Script for Habitat 2.0

[ASzot]

Motivation and Context

New episode creator and ReplicaCAD integration code for Habitat 2.0. Also new Gym API and upgrades to the interactive play script. These are the core functionalities I need to test things are working.

New components listed in detail below.

Download Rearrangment assets:
python -m habitat_sim.utils.datasets_download --uids rearrange_task_assets

Episode Generation

Running python habitat/datasets/rearrange/rearrange_generator.py --run
will generate a single episode with default config and save it locally as rearrange_ep_dataset.json.gz.
Use:

• --config to pass a custom generator config file. (See example config: habitat/datasets/rearrange/configs/test_config.yaml)
• --num-episodes to specify the number of episodes to generate.
• --out to specify the output location for the serialized dataset.
• --debug to also generate and save debugging images/videos.
• --db-output to specify the output location for debugging images/videos (default is rearrange_ep_gen_output/).

NOTE: python habitat/datasets/rearrange/rearrange_generator.py --list will print concise list of scene, object, and receptacle handles for reference when authoring a custom generator config file.

Gym API

Create Habitat environments from Gym by first import habitat_baselines.utils.gym_definitions and then env = gym.make("EnvironmentID") the environment IDs are listed below:

• HabitatGymPick-v0
• HabitatGymPickState-v0
• HabitatGymReachState-v0
• HabitatGymOpenCabState-v0
• HabitatGymCloseCabState-v0

You can also create a "render" version of the environment which has a high-resolution 3rd person camera to show what the robot with env.render() (but this is slower!). Do this using prefix HabitatGymRender, for example HabitatGymRenderPick-v0 instead of HabitatGymPick-v0.

Pass overrides to the task with the override_options named argument in gym.make. For example gym.make("HabitatGymPick-v0", override_options=["TASK_CONFIG.TASK.ACTIONS.ARM_ACTION.GRIP_CONTROLLER", "SuctionGraspAction"]).

Interactive Play Script

Example to run

python examples/interactive_play.py --cfg configs/tasks/rearrange/composite.yaml --play-task --never-end --add-ik

Some useful arguments:

• --never-end if specified the episode will never end due to a time out.
• --add-ik will change the arm action space to IK control. Note you need
  PyBullet installed for this to work.
• --play-task will add the visualization camera to the task so you can
  easily view what's going on in the play window.
• Any config argument can be changed by specifying additional arguments.

How Has This Been Tested

Added new tests:

• RL Environments tested in test/test_rearrange_task.py::test_rearrange_baseline_envs
• Tasks tested in test/test_rearrange_task.py::test_rearrange_tasks
• Interactive play script: test/test_examples.py::test_rearrange_example_modules
• Gym wrappers: test/test_gym_wrapper.py

Types of changes

• Docs change / refactoring / dependency upgrade
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have completed my CLA (see CONTRIBUTING)
• I have added tests to cover my changes.
• All new and existing tests passed.

[vincentpierre]

Not really relevant to this PR but I get the sense that configurations and datasets/scenes are often referred as their paths. I think this is going to be maybe hard to maintain on the long-term. This forces users to hard code paths at a lot of places. I think having some static registry of keys to paths might be a better approach. This way if you want to move an asset, you would just need to change it at one place (and you could also populate this registry every time an asset is downloaded).

[vincentpierre]

It looks like the source of truth is the config and if there is no GYM_OBS_KEYS in the config, then the fallback is the observation space of the environment. Is that the expected behavior?
If the observation_space is always available, why is there a GYM_OBS_KEYS in the config? (It looks redundant).
I think we should always use env.observation_space.spaces and throw an error if it is different from the config provided.

[ASzot]

Yes, this is how it works and I think this should be the desired behavior. The GYM_OBS_KEY is used for filtering out the observations that are wanted in the Gym environment.

[aclegg3]

Some thoughts after documenting/typing the code.

[aclegg3]

LGTM

addressed most changes