Skip to content

v1.2.0
Compare
Choose a tag to compare
@Mayankm96 Mayankm96 released this 20 Sep 17:44
· 82 commits to main since this release
v1.2.0
e00d625
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

👀 Overview

We leverage the new release of Isaac Sim, 4.2.0, and bring RTX-based tiled rendering, support for multi-agent environments, and introduce many bug fixes and improvements.

Additionally, we have published an example for generating rewards using an LLM based on Eureka, available here https://github.com/isaac-sim/IsaacLabEureka.

Full Changelog: v1.1.0...v1.2.0

🔆 Highlighted Features

shadowhand_cube_tiled_rendering.mp4
shadowhand_multi_agent_hand_over.mp4
deformable_teddy_pick_up.mp4

✨ New Features

  • Adds RTX-based tiled rendering. This improves the overall rendering speed and quality.
  • Adds the direct workflow perceptive Shadowhand Cube Repose environment Isaac-Repose-Cube-Shadow-Vision-Direct-v0 by @kellyguo11.
  • Adds support for multi-agent environments with the Direct workflow, with support for MAPPO and IPPO in SKRL by @Toni-SM
  • Adds the direct workflow multi-agent environments Isaac-Cart-Double-Pendulum-Direct-v0 and Isaac-Shadow-Hand-Over-Direct-v0 by @Toni-SM
  • Adds throughput benchmarking scripts for the different learning workflows by @kellyguo11 in #759
  • Adds results for the benchmarks in the documentation here for different types of hardware by @kellyguo11
  • Adds the direct workflow Allegro hand environment by @kellyguo11 in #709
  • Adds video recording to the play scripts in RL workflows by @j3soon in #763
  • Adds comparison tables for the supported RL libraries here by @kellyguo11
  • Add APIs for deformable asset by @masoudmoghani in #630
  • Adds support for MJCF converter by @qqqwan in #957
  • Adds a function to define camera configs through intrinsic matrix by @pascal-roth in #617
  • Adds configurable modifiers to observation manager by @jtigue-bdai in #830
  • Adds the Hydra configuration system for RL training by @Dhoeller19 in #700

🔧 Improvements

  • Uses PhysX accelerations for rigid body acceleration data by @Mayankm96 in #760
  • Adds documentation on the frames for asset data by @Mayankm96 in #742
  • Renames Unitree configs in locomotion tasks to match properly by @Mayankm96 in #714
  • Adds option to set the height of the border in the TerrainGenerator by @pascal-roth in #744
  • Adds a cli arg to run_all_tests.py for testing a selected extension by @jsmith-bdai in #753
  • Decouples rigid object and articulation asset classes by @Mayankm96 in #644
  • Adds performance optimizations for domain randomization by @kellyguo11 in #494
  • Allows having hybrid dimensional terms inside an observation group by @Mayankm96 in #772
  • Adds a flag to preserve joint order inside JointActionCfg action term by @xav-nal in #787
  • Adds the ability to resume training from a checkpoint with rl_games by @sizsJEon in #797
  • Adds windows configuration to VS code tasks by @johnBuffer in #963
  • Adapts A and D button bindings in the keyboard device by @zoctipus in #910
  • Uses torch.einsum for quat_rotate and quat_rotate_inverse operations by @dxyy1 in #900
  • Expands on articulation test for multiple instances and devices by @jsmith-bdai in #872
  • Adds setting of environment seed at initialization by @Mayankm96 in #940
  • Disables default viewport when headless but cameras are enabled by @kellyguo11 in #851
  • Simplifies the return type for parse_env_cfg method by @Mayankm96 in #965
  • Simplifies the if-elses inside the event manager apply method by @Mayankm96 in #948

🐛 Bug Fixes

  • Fixes rendering frame delays. Rendered images now faithfully represent the latest state of the physics scene. We added the flag
    rerender_on_reset in the environment configs to toggle an additional render step when a reset happens. When activated, the images/observation always represent the latest state of the environment, but this also reduces performance.
  • Fixes wrap_to_pi function in math utilities by @Mayankm96 in #771
  • Fixes setting of pose when spawning a mesh by @masoudmoghani in #692
  • Fixes caching of the terrain using the terrain generator by @Mayankm96 in #757
  • Fixes running train scripts when rsl_rl is not installed by @Dhoeller19 in #784, #789
  • Adds flag to recompute inertia when randomizing the mass of a rigid body by @Mayankm96 in #989
  • Fixes support for classmethod when defining a configclass by @Mayankm96 in #901
  • Fixes Sb3VecEnvWrapper to clear buffer on reset by @EricJin2002 in #974
  • Fixes venv and conda pip installation on windows by @kellyguo11 in #970
  • Sets native livestream extensions to Isaac Sim 4.1-4.0 defaults by @jtigue-bdai in #954
  • Defaults the gym video recorder fps to match episode decimation by @ozhanozen in #894
  • Fixes the event manager's apply method by @kellyguo11 in #936
  • Updates camera docs with world units and introduces new test for intrinsics by @pascal-roth in #886
  • Adds the ability to resume training from a checkpoint with rl_games by @sizsJEon in #797

💔 Breaking Changes

  • Simplifies device setting in SimulationCfg and AppLauncher by @Dhoeller19 in #696
  • Fixes conflict in teleop-device command line argument in scripts by @Dhoeller19 in #791
  • Converts container.sh into Python utilities by @hhansen-bdai in f565c33
  • Drops support for TiledCamera for Isaac Sim 4.1

✈️ Migration Guide

Setting the simulation device into the simulation context

Details

Previously, changing the simulation device to CPU required users to set other simulation parameters (such as disabling GPU physics and GPU pipelines). This made setting up the device appear complex. We now simplify the checks for device directly inside the simulation context, so users only need to specify the device through the configuration object.

Before:

sim_utils.SimulationCfg(device="cpu", use_gpu_pipeline=False, dt=0.01, physx=sim_utils.PhysxCfg(use_gpu=False))

Now:

sim_utils.SimulationCfg(device="cpu", dt=0.01, physx=sim_utils.PhysxCfg())

Setting the simulation device from CLI

Details

Previously, users could specify the device through the command line argument --device_id. However, this made it ambiguous when users wanted to set the device to CPU. Thus, instead of the device ID, users need to specify the device explicitly through the argument --device. The valid options for the device name are:

  • "cpu": runs simulation on CPU
  • "cuda": runs simulation on GPU with device ID at default index
  • "cuda:N": runs simulation on GPU with device ID at N. For instance, "cuda:0" will use device at index "0".

Due to the above change, the command line interaction with some of the scripts has changed.

Before:

./isaaclab.sh -p source/standalone/workflows/sb3/train.py --task Isaac-Cartpole-v0 --headless --cpu

Now:

./isaaclab.sh -p source/standalone/workflows/sb3/train.py --task Isaac-Cartpole-v0 --headless --device cpu

Renaming of teleoperation device CLI in standalone scripts

Details

Since --device is now an argument provided by the AppLauncher, it conflicted with the command-line argument used for specifying the teleoperation-device in some of the standalone scripts. Thus, to fix this conflict, the teleoperation-device now needs to be specified through --teleop_device argument.

Before:

./isaaclab.sh -p source/standalone/environments/teleoperation/teleop_se3_agent.py --task Isaac-Lift-Cube-Franka-IK-Rel-v0 --num_envs 1 --device keyboard

Now:

./isaaclab.sh -p source/standalone/environments/teleoperation/teleop_se3_agent.py --task Isaac-Lift-Cube-Franka-IK-Rel-v0 --num_envs 1 --teleop_device keyboard

Using Python-version of container utility script

Details

The prior container.sh became quite complex as it had many different use cases in one script. For instance, building a docker image for "base" or "ros2", as well as cluster deployment. As more users wanted to have the flexibility to overlay their own docker settings, maintaining this bash script became cumbersome. Hence, we migrated its features into a Python script in this release. Additionally, we split the cluster-related utilities into their own script inside the docker/cluster directory.

We still maintain backward compatibility for container.sh. Internally, it calls the Python script container.py. We request users to use the Python script directly.

Before:

./docker/container.sh start

Now:

./docker/container.py start

Using separate directories for logging videos in RL workflows

Details

Previously, users could record videos during the RL training by specifying the --video flag to the train.py script. The videos would be saved inside the videos directory in the corresponding log directory of the run.

Since many users requested to also be able to record videos while inferencing the policy, recording videos have also been added to the play.py script. Since changing the prefix of the video file names is not possible, the videos from the train and play scripts are saved inside the videos/train and videos/play directories, respectively.

Drops support for the tiled camera with Isaac Sim 4.1

Details

Various fixes have been made to the tiled camera rendering pipeline in Isaac Sim 4.2. This made supporting the tiled camera with Isaac Sim 4.1 difficult. Hence, for the best experience, we advice switching to Isaac Sim 4.2 with this release of Isaac Lab.

🤗 New Contributors

Contributors

bearpaw, kellyguo11, and 22 other contributors
Assets 2
Loading
0 Join discussion