Extreme Parkour with Legged Robots

Xuxin Cheng, Kexin Shi, Ananye Agarwal, Deepak Pathak

Seminal September 2023

The robot traverses an indoor course of boxes, gaps and ramps in one run.

TL;DR One neural network maps depth from a single front camera on a low-cost Unitree A1 straight to joint commands and picks its own heading. Trained with RL in simulation, the robot high-jumps 2x its height, long-jumps 2x its length, crosses 37° tilted ramps and walks in a handstand.

What they do

Parkour on a low-cost Unitree A1 with imprecise actuation and one front-facing depth camera that is low-frequency, jittery and prone to artifacts. Instead of building an elevation map and planning, a single network goes from depth and proprioception to joint commands, and it also chooses its own heading: in long or high jumps a few degrees of heading error leads to failure, so human-given directions are not good enough.

How it works

Unified inner-product reward. Waypoints on the terrain define a direction, and the reward tracks velocity along it in the world frame (in the base frame the robot learned to turn around obstacles). Foot contacts within 5 cm of an edge are penalized. A switchable term aligning the body’s forward axis with straight down produces the handstand.
Phase 1. RL in simulation from privileged scandots, waypoint heading and environment parameters (estimated with ROA), on tilted ramps, gaps, hurdles and high steps in an automatic difficulty curriculum.
Phase 2, dual distillation. A convnet-GRU on depth replaces the scandots and is trained with DAgger on the phase-1 actions, while a heading predictor learns the oracle yaw. During training the student observes a mixture of teacher and predicted headings (MTS) to avoid distribution drift.
Deployment. 58x87 depth at about 10 Hz and the policy at 50 Hz on a Jetson NX; training takes under 20 h on one 3090.

Results

Real robot: high jump onto a 0.5 m box (2x hip height), long jump over a 0.8 m gap (2x the feet separation), tilted ramps up to 37°, and handstand walking on grass and down stairs from proprioception alone.
Real world, 5 trials per terrain and difficulty: 20-80% higher success than the baselines on the hardest instance of each terrain.
Simulation, 256 robots: mean x-displacement 0.98 with edge violation 0.03, versus 0.75 without the inner-product reward (0.14 on steps) and 0.82 with 0.20 edge violation for a noisy elevation map.
Distillation reaches 0.92 displacement against 0.94 with oracle heading; always feeding predicted heading gives 0.12, masking it 0.05.

Limitations

The paper has no limitations section. Everything runs on one robot, the A1; real-world comparisons use 5 trials per setting. The handstand policy uses no vision. Mobile manipulation is named as future work.

Videos

Long jump across a 0.8 m gap, twice the robot's length.

Handstand walking on the front two legs on outdoor grass.

Figures

From Figure 1: A low-cost robot with imprecise actuation long-jumps gaps 2x its length, high-jumps obstacles 2x its h... — From Figure 1: A low-cost robot with imprecise actuation long-jumps gaps 2x its length, high-jumps obstacles 2x its height, runs over tilted ramps and walks on its front two legs, all with one network operating on depth from a single front-facing camera, without explicit mapping or planning.

From Figure 2: Training overview. Phase 1 learns a policy with RL from privileged scandots, environment parameters (r... — From Figure 2: Training overview. Phase 1 learns a policy with RL from privileged scandots, environment parameters (recovered from observation history with ROA) and waypoint heading; phase 2 distills it into a policy that runs from onboard depth and decides its own heading for each obstacle.

From Figure 7: Real-world success rate over 5 trials on hurdles, steps, gaps and tilted ramps of increasing size. Our... — From Figure 7: Real-world success rate over 5 trials on hurdles, steps, gaps and tilted ramps of increasing size. Ours is 20-80% higher on the hardest instance of each terrain; NoDir (heading from a human joystick) fails on tilted ramps, and NoClear (no edge penalty) steps close to edges and often falls.

Abstract

Humans can perform parkour by traversing obstacles in a highly dynamic fashion requiring precise eye-muscle coordination and movement. Getting robots to do the same task requires overcoming similar challenges. Classically, this is done by independently engineering perception, actuation, and control systems to very low tolerances. This restricts them to tightly controlled settings such as a predetermined obstacle course in labs. In contrast, humans are able to learn parkour through practice without significantly changing their underlying biology. In this paper, we take a similar approach to developing robot parkour on a small low-cost robot with imprecise actuation and a single front-facing depth camera for perception which is low-frequency, jittery, and prone to artifacts. We show how a single neural net policy operating directly from a camera image, trained in simulation with large-scale RL, can overcome imprecise sensing and actuation to output highly precise control behavior end-to-end. We show our robot can perform a high jump on obstacles 2x its height, long jump across gaps 2x its length, do a handstand and run across tilted ramps, and generalize to novel obstacle courses with different physical properties. Parkour videos at https://extreme-parkour.github.io/

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong