Robot Parkour Learning

Ziwen Zhuang, Zipeng Fu, Jianren Wang, Christopher Atkeson, Soeren Schwertfeger, Chelsea Finn, Hang Zhao

Seminal September 2023

An A1 leaps across the gap between two desks.

TL;DR Five parkour skills are trained in simulation with RL that first lets the robot pass through obstacles and then enforces real contact, then distilled into one depth-camera policy for low-cost quadrupeds. On a real A1 it climbs 0.40 m obstacles and leaps 0.60 m gaps with 80% success.

What they do

Parkour on low-cost quadrupeds (Unitree A1 and Go1): climbing, leaping gaps, crawling under barriers, tilting through narrow slits and running, all from one end-to-end policy that uses only onboard depth and proprioception. Every skill is learned with the same simple reward (move forward at about 1 m/s, an energy penalty, an alive bonus) and no reference motion data.

How it works

Skill policies. One GRU policy per skill, trained with PPO in IsaacGym using privileged obstacle information (distance, height, width, type) and randomized physics.
Soft-then-hard dynamics. Inspired by direct collocation, obstacles start out penetrable. A penalty on penetration volume and depth at sampled collision points (denser around the motors), together with an automatic difficulty curriculum, gradually enforces contact. Fine-tuning then uses solid obstacles.
Distillation. DAgger distills the five skills into one GRU policy on a 48x64 depth image, with the teacher chosen by obstacle type.
Deployment. Depth images are processed so sim and real look alike (noise and artifacts in sim; hole filling and smoothing on the robot). Depth arrives at 10 Hz, the policy runs at 50 Hz, and torque is capped at 25 Nm.

Results

Soft-dynamics pre-training reaches about 95% success across the four obstacle skills in simulation. RND and hard-dynamics-only training never learn to climb or leap.
Distilled policy on harder test terrains in simulation: 86% climb, 80% leap, 100% crawl, 73% tilt. Blind, MLP and No Distill variants score at most 1% on climbing and leaping.
Real A1, 10 trials per setting: 0.40 m climbs (1.53x robot height) at 80%, 0.60 m gaps at 80%, 0.2 m crawls at 90%, and 0.28 m slits by tilting. It does best on every skill against a blind policy and the built-in MPC controllers.
Outdoors it leaps a 0.6 m gap and climbs consecutive 0.42 m steps. When a climb fails, it backs off and tries again, a behavior nobody programmed.

Limitations

Simulation environments are built by hand, so each new skill needs new obstacles added to simulation. The policy sees depth only, not RGB.

Videos

An A1 climbs onto a ledge outdoors at night.

A Go1 trained with the same pipeline leaps between desks.

Figures

From Figure 1: One end-to-end vision-based policy lets a low-cost robot climb high obstacles, leap over large gaps, c... — From Figure 1: One end-to-end vision-based policy lets a low-cost robot climb high obstacles, leap over large gaps, crawl beneath low barriers, squeeze through thin slits and run.

From Figure 3: Each skill under soft dynamics constraints, where the obstacles are penetrable, and under hard dynamic... — From Figure 3: Each skill under soft dynamics constraints, where the obstacles are penetrable, and under hard dynamics constraints.

From Figure 4: Collision points sampled on the robot body; points that penetrate an obstacle are red.

From Figure 6: Real-world indoor success rates as climb, leap, crawl and tilt obstacles get harder; the parkour polic... — From Figure 6: Real-world indoor success rates as climb, leap, crawl and tilt obstacles get harder; the parkour policy beats a blind policy and the A1's built-in MPC controllers.

Abstract

Parkour is a grand challenge for legged locomotion that requires robots to overcome various obstacles rapidly in complex environments. Existing methods can generate either diverse but blind locomotion skills or vision-based but specialized skills by using reference animal data or complex rewards. However, autonomous parkour requires robots to learn generalizable skills that are both vision-based and diverse to perceive and react to various scenarios. In this work, we propose a system for learning a single end-to-end vision-based parkour policy of diverse parkour skills using a simple reward without any reference motion data. We develop a reinforcement learning method inspired by direct collocation to generate parkour skills, including climbing over high obstacles, leaping over large gaps, crawling beneath low barriers, squeezing through thin slits, and running. We distill these skills into a single vision-based parkour policy and transfer it to a quadrupedal robot using its egocentric depth camera. We demonstrate that our system can empower two different low-cost robots to autonomously select and execute appropriate parkour skills to traverse challenging real-world environments.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong