awesome-humanoid-manipulation

A curated list of awesome papers and resources on humanoid manipuation, or upper-body humanoid robot learning. This repo is inspired by awesome-humanoid-learning.

Note: We also included papers on the relevant topics:

1. Bimanual multi-fingered manipulation
2. Dual-arm manipulation with other end effectors (#1)
3. humanoid or digital human manipulation in physically based animation.

---

Contents

• Humanoid Models
  • Humanoids
  • Dexterous Hands
  • Dual-Arm Robots
• Workshops
• Projects
• Papers
  • Humanoid Robot Manipulation
  • Bimanual Dexterous Multi-Fingered Manipulation
  • Dual-Arm Manipulation with Other End Effectors
  • Physically Simulated HOI Animations

Robot Models

Humanoids

Name	Maker	Formats	License	Meshes	Inertias	Collisions
H1	Unitree	URDF & MJCF, USD	BSD-3-Clause	✔️	✔️	✔️
G1	Unitree	URDF & MJCF	BSD-3-Clause	✔️	✔️	✔️
GR-1	FFTAI	URDF, MJCF	GPL-3.0	✔️	✔️	✔️
Digit	Agility Robotics	URDF	✖️	✔️	✔️	✔️

Dexterous Hands

Name	Maker	Formats	License	Meshes	Inertias	Collisions
Ability Hand	PSYONIC, Inc.	MJCF, URDF	✖️	✔️	✔️	✖️
Allegro Hand	Wonik Robotics	URDF, MJCF	BSD	✔️	✔️	✔️
Shadow Hand	Shadow Robot	URDF, MJCF	BSD	✔️	✔️	✔️

Dual-Arm Robots

Name	Maker	Formats	License	Visuals	Inertias	Collisions
YuMi	ABB	URDF	BSD-2-Clause	✔️	✔️	✔️
Dual iiwa 14	KUKA	URDF, Xacro	BSD-3-Clause	✔️	✔️	✔️

Workshops

CORL 2024 [Workshop on Whole-body Control and Bimanual Manipulation: Applications in Humanoids and Beyond]

Projects

Lerobot [LeRobot: State-of-the-art AI for real-world robotics]

Papers

YYYY.MM is the date when paper appears on arxiv.org (if available).

Humanoid Robot Manipulation

• [2024.10] EgoMimic: Scaling Imitation Learning via Egocentric Video [IL] [project] [paper] [code]

• [2024.10] DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning [IL] [project] [paper]

• [2024.10] OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitation [IL] [project] [paper]

• [2024.10] Generalizable Humanoid Manipulation with Improved 3D Diffusion Policies [IL] [diffusion] [project] [paper] [code(Learning)][code(Teleop)]

• [2024.08] RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual Dexterous Robot Hands [IL] [project] [paper] [code]

• [2024.07] Open-TeleVision Teleoperation with Immersive Active Visual Feedback [IL] [project] [paper] [code]

• [2024.07] GRUtopia: Dream General Robots in a City at Scale [benchmark] [doc] [paper] [project&code]

• [2024.07] BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark [benchmark] [project] [paper] [code]

• [2024.06] HumanPlus: Humanoid Shadowing and Imitation from Humans [IL] [project] [paper] [code]

• [2024.06] Learning Hybrid Behavior Planning for Autonomous Loco-manipulation [VLM] [project] [paper]

• [2024.06] OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning [benchmark] [project] [paper] [code]

• [2024.04] Large Language Models for Orchestrating Bimanual Robots [LLM] [paper] [website], [code]

• [2024.03] HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation [benchmark] [project] [paper] [code]

Bimanual Dexterous Multi-Fingered Manipulation

• [2024.11] Object-Centric Dexterous Manipulation from Human Motion Data [RL] [project] [paper]

• [2024.10] Learning Diverse Bimanual Dexterous Manipulation Skills From Human [IL] [project] [paper]

• [2024.08] ACE: A Cross-platform Visual-Exoskeletons for Low-Cost Dexterous Teleoperation [teleop] [project] [paper][code]

• [2024.07] Bunny-Vision Pro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning [IL] [project] [paper][code]

• [2024.04] Learning Visuotactile Skills with Two Multifingered Hands [IL] [touch] [project] [paper][code]

• [2024.03] DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation [IL] [project] [paper][code]

• [CORL 2024] Bimanual Dexterity for Complex Tasks [IL] [project] [paper]

Dual-Arm Manipulation with Other End Effectors

• [2024.11] AsymDex: Leveraging Asymmetry and Relative Motion in Learning Bimanual Dexterity [RL] [paper] [project]

• [2024.10] RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation [IL] [github] [paper] [project]

• [2024.10] ALOHA Unleashed: A Simple Recipe for Robot Dexterity [IL] [project] [paper]

• [2024.09] ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation [VLM] [project] [paper][code]

• [2024.07] VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulation [IL] [VLM] [project] [paper][code]

• [2024.07] PerAct2: Benchmarking and Learning for Robotic Bimanual Manipulation Tasks [IL] [project] [paper][code]

• [2024.01] Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation [IL] [project] [paper][code(learning)] [code(hardware)]

Physically Simulated Humanoid Animations and Digital Human-Object Interaction

• [2024.10] Autonomous Character-Scene Interaction Synthesis from Text Instruction [mocap] [project] [paper]

• [2024.07] Grasping Diverse Objects with Simulated Humanoids [RL] [project] [paper]

• [2024.06] Human-Object Interaction from Human-Level Instructions [LLM] [project] [paper]

• [2024.06] CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics [RL] [project] [paper]

• [2024.06] CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangement [mocap] [project] [paper][code]

• [2024.04] HOI-M3: Capture Multiple Humans and Objects Interaction within Contextual Environment [mocap] [project] [paper][code]

• [2024.03] AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agents [RL] [project] [paper][code]