Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation

Tairan He, Zhengyi Luo, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu, Guanya Shi

★ Must read March 2024

The humanoid follows the teleoperator's left-right kick and strikes two balls.

TL;DR H2O trains one RL policy that makes a Unitree H1 humanoid copy a human's whole-body motion in real time from a single RGB camera, after a privileged simulated imitator filters the retargeted AMASS motions down to the ~8.5k of 10k the robot can perform. In simulation it tracks 72.5% of the retargeted motions, versus 67.9% when trained without that filtering.

What they do

H2O teleoperates a full-sized Unitree H1 humanoid, whole body including the legs, in real time from a single RGB webcam. No exoskeleton, markers or force sensors on the operator. The main obstacle they identify is data: many human motions are infeasible for the robot, so they clean a large mocap dataset in simulation before training a sim-to-real RL tracking policy on it.

How it works

Retargeting. Fit an SMPL body shape to the H1 using 12 corresponding joints, then retarget AMASS motions by matching those joints; 13k sequences become 10k.
Sim-to-data. Train a privileged imitator (full rigid-body state, no domain randomization) on all 10k and discard what it fails to track, leaving about 8.5k feasible sequences.
Teleoperation policy. PPO policy outputs 19 joint targets for a PD controller. Its goal input is 8 keypoints (shoulders, elbows, hands, ankles), their offset from the robot’s own keypoints, and their velocities. Trained with domain randomization (friction, mass, CoM, PD gains, 20-60 ms delay, pushes, rough terrain).
Deployment. HybrIK estimates the human pose at 30 Hz from a 1080p webcam. Robot linear velocity comes from motion capture.

Results

On the 10k retargeted motions in simulation, H2O succeeds on 72.5%, versus 67.9% without sim-to-data and 53.2% with a goal of keypoint positions only. The privileged policy, which cannot transfer to the real robot, reaches 85.5%.
Global MPJPE is 166.7 mm for H2O versus 176.6 mm without sim-to-data. Training on 0.1% of the clean data still gives 52.0% success.
On the real robot: walking, back jumping, ball kicking, catching and binning a box, stroller pushing and boxing, and it keeps its balance when kicked.

Limitations

The real system needs motion capture for the robot’s linear velocity. RGB pose estimation adds latency and error, and the operator gets only visual feedback. There is no systematic way to decide which motions are feasible for a given robot, and richer goal representations cost RL sample efficiency.

Videos

Back jumping, copied from the teleoperator in real time.

The humanoid steps and punches along with a boxing teleoperator.

Figures

From Figure 1: The humanoid teleoperated in real time from an RGB camera. (a) It steps forward and punches a box out ... — From Figure 1: The humanoid teleoperated in real time from an RGB camera. (a) It steps forward and punches a box out of place, then makes a victory gesture. (b) It sidesteps to line up with a ball and kicks it with its right foot. (c) It walks forward pushing a stroller. (d) It catches a box, rotates its waist and drops the box into a waste bin.

From Figure 4: Overview of H2O. (a) Retargeting: fit the SMPL body shape to the humanoid, retarget the motions, then ... — From Figure 4: Overview of H2O. (a) Retargeting: fit the SMPL body shape to the humanoid, retarget the motions, then remove infeasible ones with a privileged imitation policy to get a clean dataset. (b) Sim-to-real training: an imitation policy learns to track motion goals sampled from the clean dataset. (c) Deployment: an RGB camera and pose estimator capture the human, and the trained policy makes the humanoid mimic them.

From Figure 7: Robustness tests with strong kicks to the robot. The policy keeps its balance during both stable and d... — From Figure 7: Robustness tests with strong kicks to the robot. The policy keeps its balance during both stable and dynamic teleoperated motions.

Abstract

We present Human to Humanoid (H2O), a reinforcement learning (RL) based framework that enables real-time whole-body teleoperation of a full-sized humanoid robot with only an RGB camera. To create a large-scale retargeted motion dataset of human movements for humanoid robots, we propose a scalable "sim-to-data" process to filter and pick feasible motions using a privileged motion imitator. Afterwards, we train a robust real-time humanoid motion imitator in simulation using these refined motions and transfer it to the real humanoid robot in a zero-shot manner. We successfully achieve teleoperation of dynamic whole-body motions in real-world scenarios, including walking, back jumping, kicking, turning, waving, pushing, boxing, etc. To the best of our knowledge, this is the first demonstration to achieve learning-based real-time whole-body humanoid teleoperation.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong