Matt Strong

Matt Strong

CS PhD at Stanford University

← Paper Digest

ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills

Tairan He, Jiawei Gao, Wenli Xiao, Yuanhang Zhang, Zi Wang, Jiashun Wang, Zhengyi Luo, Guanqi He, Nikhil Sobanbabu, Chaoyi Pan, Zeji Yi, Guannan Qu, Kris Kitani, Jessica Hodgins, Linxi "Jim" Fan, Yuke Zhu, Changliu Liu, Guanya Shi

Seminal February 2025

Highlight reel of the real Unitree G1 performing agile motions taken from human videos.

TL;DR ASAP learns a residual "delta action" model from real humanoid rollouts so the simulator reproduces real dynamics, then fine-tunes motion-tracking policies inside it. On a Unitree G1 this cuts global tracking error from 61.2 to 50.2 mm on a kick and from 159 to 112 mm on an unseen motion.

What they do

Agile humanoid skills break on hardware because simulated and real dynamics differ. ASAP runs simulation-trained motion-tracking policies on a Unitree G1 and uses the recorded trajectories to learn a residual action model that makes the simulator behave like the real robot. The policy is then fine-tuned in that corrected simulator.

How it works

  • Pretraining. Human videos are converted to SMPL motions with TRAM, filtered by physics-based tracking and retargeted to the 23-DoF G1. A PPO policy tracks each motion from proprioception and a time phase alone, so deployment needs no odometry.
  • Delta action model. Real actions are replayed in simulation from real states. An RL policy adds a correction to each action and is rewarded for matching the next real state, measured with motion capture. On hardware only the 4 ankle joints are corrected, learned from 100 clips.
  • Fine-tuning. The pretrained policy is fine-tuned in the simulator with the frozen delta model, then deployed without it.

Results

  • Open-loop replay over 1.0 s (IsaacGym to IsaacSim): 37.9 mm global error, against 77.6 mm for SysID and 68.1 mm for a learned delta dynamics model.
  • Hard motions in Genesis: 100% success and 129 mm error, against 175 mm with no fine-tuning, 186 mm for SysID, and 190 mm with 60% success for delta dynamics.
  • Real G1: error drops from 61.2 to 50.2 mm on a kick and from 159 to 112 mm on an out-of-distribution motion.
  • Fine-tuning with random action noise instead reaches 150 mm, against 126 mm for ASAP.

Limitations

The motions overheat the motors, and two G1 robots broke during data collection. The pipeline needs motion capture. A full 23-DoF delta model needs more than 400 clips, so the hardware experiments correct only the ankles. Real-world numbers cover two motions.

Videos

The real G1 tracking a crouching basketball celebration motion.
The real G1 tracking a one-legged fadeaway jump-shot motion.

Figures

From Figure 2: Overview of ASAP. (a) Motion-tracking policies pretrained on motions retargeted from human videos roll...
From Figure 2: Overview of ASAP. (a) Motion-tracking policies pretrained on motions retargeted from human videos roll out real-world trajectories. (b) A delta action model is trained to minimize the gap between simulated and real states. (c) The frozen delta action model is put into the simulator and the pretrained policy is fine-tuned. (d) The fine-tuned policy is deployed on the real robot without the delta action model.
From Figure 4: The compared approaches: (a) model-free RL training, (b) system identification from real-world data, (...
From Figure 4: The compared approaches: (a) model-free RL training, (b) system identification from real-world data, (c) a learned delta dynamics model, (d) ASAP, a learned delta action model.
From Figure 8: The G1 before (top, blue) and after (bottom, red) ASAP fine-tuning, on in-distribution motions (left) ...
From Figure 8: The G1 before (top, blue) and after (bottom, red) ASAP fine-tuning, on in-distribution motions (left) and out-of-distribution motions (right). After fine-tuning the robot moves more smoothly, with less jerky lower-body motion.
Abstract

Humanoid robots hold the potential for unparalleled versatility in performing human-like, whole-body skills. However, achieving agile and coordinated whole-body motions remains a significant challenge due to the dynamics mismatch between simulation and the real world. Existing approaches, such as system identification (SysID) and domain randomization (DR) methods, often rely on labor-intensive parameter tuning or result in overly conservative policies that sacrifice agility. In this paper, we present ASAP (Aligning Simulation and Real-World Physics), a two-stage framework designed to tackle the dynamics mismatch and enable agile humanoid whole-body skills. In the first stage, we pre-train motion tracking policies in simulation using retargeted human motion data. In the second stage, we deploy the policies in the real world and collect real-world data to train a delta (residual) action model that compensates for the dynamics mismatch. Then, ASAP fine-tunes pre-trained policies with the delta action model integrated into the simulator to align effectively with real-world dynamics. We evaluate ASAP across three transfer scenarios: IsaacGym to IsaacSim, IsaacGym to Genesis, and IsaacGym to the real-world Unitree G1 humanoid robot. Our approach significantly improves agility and whole-body coordination across various dynamic motions, reducing tracking error compared to SysID, DR, and delta dynamics learning baselines. ASAP enables highly agile motions that were previously difficult to achieve, demonstrating the potential of delta action learning in bridging simulation and real-world dynamics. These results suggest a promising sim-to-real direction for developing more expressive and agile humanoids.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.