ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robots

David Hoeller, Nikita Rudin, Dhionis Sako, Marco Hutter

Seminal June 2023

TL;DR A navigation policy picks among five learned locomotion skills (walk, jump, climb up, climb down, crouch) and steers them using a learned 3D reconstruction of the terrain, so ANYmal D can cross parkour-style box courses. The pipeline is trained only in simulation, reaches up to 2 m/s on the real robot, and succeeds in 96-98% of simulated courses.

What they do

A fully learned pipeline gets a quadruped to a target across parkour-like obstacles (boxes, gaps, tables, stairs) within a time limit. It uses no expert demonstrations, offline planning or prior map. Everything is trained in simulation and deployed on ANYmal D.

How it works

Perception. Two encoder-decoder 3D CNNs work on voxelized point clouds from six depth cameras and a LiDAR. A coarse network (12.5 cm voxels, 4 m map) feeds its previous output back in to remember occluded surfaces. A fine network (6.25 cm voxels, 2 m map) refines the area near the robot.
Locomotion. Five RL skills run at 50 Hz: walk, jump, climb up, climb down and crouch. Each must reach a commanded position and heading within a given time, using a 2.5D map taken from the reconstruction. Training adds front-back and left-right symmetry augmentation.
Navigation. A 5 Hz RL policy runs with the frozen skills in the loop. It sees the goal, the remaining time and the perception latent. A modified PPO gives it a hybrid output: a categorical skill choice plus Gaussian position, heading and time commands. Reward is sparse, given at the end of the episode.

Results

On hardware the robot reaches 2 m/s and climbs a 0.9 m platform. It also stands up after falling off a box and re-routes when obstacles are pulled away.
Every skill holds up to about 90% of its training difficulty: gaps and boxes up to 1 m, and passages down to 0.4 m.
In simulation over 1000 roll-outs per terrain, the learned navigation succeeds 98.2/96.3/97.6% of the time. Hand-coded skills and targets reach 95.3/60.9/75.3%. Removing the heading and time outputs drops success to as low as 71.9%.
The policy climbs down directly from boxes up to 1 m high. Above that it increasingly takes a longer, safer route.

Limitations

Tested on a limited set of box-based scenarios, and generalization to new obstacles is untested. The eight networks are tuned separately and depend on each other, so changing one skill means retraining perception and navigation. Navigation only converges with a dedicated curriculum.

Videos

Figures

From Figure 1: ANYmal D running the full pipeline indoors and outdoors, performing dynamic maneuvers on boxes and tou... — From Figure 1: ANYmal D running the full pipeline indoors and outdoors, performing dynamic maneuvers on boxes and touching them with its limbs where needed.

From Figure 2: The three modules. Perception turns point clouds into a scene latent and a map, locomotion holds the l... — From Figure 2: The three modules. Perception turns point clouds into a scene latent and a map, locomotion holds the low-level skills, and navigation uses the latent and the target to plan a path and select a skill.

From Figure 4: Training scenarios and learned behaviors for jumping, climbing down, climbing up, crouching and walkin... — From Figure 4: Training scenarios and learned behaviors for jumping, climbing down, climbing up, crouching and walking, with the success rate of each skill as obstacle difficulty rises past the training range.

From Figure 5: Adaptive path selection. The likelihood of going up or down along the direct path drops as the box get... — From Figure 5: Adaptive path selection. The likelihood of going up or down along the direct path drops as the box gets taller, and on the real robot the policy takes a different route to the same target for 0.75 m and 1.15 m boxes.

Abstract

Performing agile navigation with four-legged robots is a challenging task due to the highly dynamic motions, contacts with various parts of the robot, and the limited field of view of the perception sensors. In this paper, we propose a fully-learned approach to train such robots and conquer scenarios that are reminiscent of parkour challenges. The method involves training advanced locomotion skills for several types of obstacles, such as walking, jumping, climbing, and crouching, and then using a high-level policy to select and control those skills across the terrain. Thanks to our hierarchical formulation, the navigation policy is aware of the capabilities of each skill, and it will adapt its behavior depending on the scenario at hand. Additionally, a perception module is trained to reconstruct obstacles from highly occluded and noisy sensory data and endows the pipeline with scene understanding. Compared to previous attempts, our method can plan a path for challenging scenarios without expert demonstration, offline computation, a priori knowledge of the environment, or taking contacts explicitly into account. While these modules are trained from simulated data only, our real-world experiments demonstrate successful transfer on hardware, where the robot navigates and crosses consecutive challenging obstacles with speeds of up to two meters per second. The supplementary video can be found on the project website: https://sites.google.com/leggedrobotics.com/agile-navigation

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong