HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation

Carmelo Sferrazza, Dun-Ming Huang, Xingyu Lin, Youngwoon Lee, Pieter Abbeel

Seminal March 2024

The highbar task, hanging from a horizontal bar and swinging the whole body.

TL;DR A MuJoCo benchmark of 27 tasks (12 locomotion, 15 whole-body manipulation) for a Unitree H1 humanoid with two Shadow Hands and a 61-D action space. DreamerV3, TD-MPC2, SAC and PPO stay below the success threshold on most tasks, and every method completes 0 of 4 kitchen subtasks.

What they do

HumanoidBench is a MuJoCo benchmark for whole-body humanoid control: a Unitree H1 with two Shadow Hands (forearms removed), 12 locomotion tasks and 15 whole-body manipulation tasks such as truck unloading, door opening and basketball. Four standard RL algorithms are benchmarked on every task, plus a hierarchical baseline.

How it works

Robot and control. 61-D action space (19 body, 21 per hand) as joint position targets at 50 Hz; the robot state is 151-D. Other robots (Unitree G1, Agility Digit) and end-effectors are also provided.
Sensing. Two head cameras and whole-body tactile sensing with 448 taxels giving 3D contact forces, denser at the hands. The benchmark itself uses only proprioception and object states, with the same robot observations in every task.
Tasks. Each task mixes dense shaping and sparse subtask rewards, with a return threshold marking success. The default model runs at 1050 FPS on one CPU.
Hierarchical baseline. One- and two-hand reaching policies are trained with PPO in MuJoCo MJX on 32,768 parallel environments (2B steps in 36 h, 4B steps in 60 h) under force perturbations, then frozen. DreamerV3 or TD-MPC2 learns a high-level policy that outputs reaching targets.

Results

With about 48 h per run (10M steps for DreamerV3 and SAC, 2M for TD-MPC2), all baselines stay below the success threshold on most tasks. Locomotion is easiest: the best maximum return on walk is 932 (DreamerV3) against a target of 700, while SAC reaches 69.
Long-horizon manipulation is mostly unsolved: kitchen scores 0 for every method (target 4), highbar peaks at 83 (target 750) and truck at 1674 (target 3000).
On walk, removing the hands or zeroing the 42 hand actuators (61-D to 19-D actions) makes learning significantly faster, so most of the drop comes from action dimensionality.
The hierarchical policy clearly beats flat RL on push, with very high success for DreamerV3, but helps less on package, where it gets close to the box but fails to lift it.

Limitations

Only state-based learning is benchmarked; vision and touch are left to future work. The forearm-less hands are not a realistic hardware model, sim-to-real is not studied, and the hierarchical approach is tested on only two tasks.

Videos

The crawl task, moving forward through a low tunnel.

The sit task, sitting down on a chair placed behind the robot.

Figures

From Figure 1: The H1 humanoid with two dexterous hands in four HumanoidBench tasks: cabinets, truck unloading, maze ... — From Figure 1: The H1 humanoid with two dexterous hands in four HumanoidBench tasks: cabinets, truck unloading, maze navigation and kitchen.

From Figure 1: Egocentric camera views (top left) and whole-body tactile readings (right) while the humanoid carries ... — From Figure 1: Egocentric camera views (top left) and whole-body tactile readings (right) while the humanoid carries a package in the truck environment. The two head cameras are highlighted in green; red marks strong pressure and yellow mild pressure.

From Figure 7: Hierarchical learning pipeline. A low-level reaching policy is pretrained in MJX and frozen, then a hi... — From Figure 7: Hierarchical learning pipeline. A low-level reaching policy is pretrained in MJX and frozen, then a high-level policy is learned on top of it.

Abstract

Humanoid robots hold great promise in assisting humans in diverse environments and tasks, due to their flexibility and adaptability leveraging human-like morphology. However, research in humanoid robots is often bottlenecked by the costly and fragile hardware setups. To accelerate algorithmic research in humanoid robots, we present a high-dimensional, simulated robot learning benchmark, HumanoidBench, featuring a humanoid robot equipped with dexterous hands and a variety of challenging whole-body manipulation and locomotion tasks. Our findings reveal that state-of-the-art reinforcement learning algorithms struggle with most tasks, whereas a hierarchical learning approach achieves superior performance when supported by robust low-level policies, such as walking or reaching. With HumanoidBench, we provide the robotics community with a platform to identify the challenges arising when solving diverse tasks with humanoid robots, facilitating prompt verification of algorithms and ideas. The open-source code is available at https://humanoid-bench.github.io.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong