Learning robust perceptive locomotion for quadrupedal robots in the wild

Takahiro Miki, Joonho Lee, Jemin Hwangbo, Lorenz Wellhausen, Vladlen Koltun, Marco Hutter

Seminal January 2022

TL;DR A simulation-trained ANYmal controller fuses noisy elevation-map height samples with proprioception through a gated recurrent encoder, so it uses the map when it is reliable and falls back on proprioception when it is not. Deployed zero-shot, it finished a 2.2 km Alpine hike in 78 minutes without a fall, close to the 76 minutes a hiking planner gives for humans.

What they do

Seeing terrain before stepping on it lets a legged robot move faster, but elevation maps built from depth sensors fail on snow, vegetation, reflective or transparent surfaces, occlusion and pose-estimation drift, so the most robust controllers to date use proprioception alone. This paper trains one ANYmal C controller that learns how far to trust the map, planning ahead when it is informative and walking proprioceptively when it is not.

How it works

Teacher. PPO in RaiSim trains an MLP with privileged inputs (noiseless height samples around each foot, contacts, friction, external forces) on randomized terrain that includes four types of stairs, with an adaptive terrain curriculum. Actions are per-leg phase offsets and residual joint targets on a CPG-style foot trajectory.
Student. Sees only proprioception and corrupted height samples: points are shifted and heights perturbed per point, per foot and per episode, in nominal, large-offset and large-noise conditions mixed 60/30/10%.
Belief encoder. A 2-layer GRU forms a belief state, and a learned sigmoid gate sets how much exteroceptive information enters it. The loss combines behavior cloning of teacher actions with reconstruction of the noiseless heights and privileged state.
Deployment. Zero-shot, no fine-tuning. A GPU elevation map updates at 20 Hz from two dome LiDARs or four RealSense cameras; the policy runs at 50 Hz.

Results

Alpine hike: 2.2 km, 120 m elevation gain, inclines up to 38%. Summit in 31 min (signage: 35), full loop in 78 min (planner: 76), with no falls and no help beyond reattaching a shoe and swapping batteries.
Steps (10 trials per height): the proprioceptive baseline starts failing at 20 cm; this controller reliably climbs up to 30.5 cm.
Speed: 1.2 vs 0.6 m/s forward and lateral, 3 vs 0.6 rad/s turning. On the obstacle course the baseline got stuck on all three obstacles.
Zero falls across all field deployments; with its sensors covered, the robot still climbed stairs after bumping the first riser.

Limitations

Uncertainty is used only implicitly: when occlusion hides a cliff edge or stepping stone, the policy assumes a continuous surface and may step off and fall. The 2.5D elevation map drops material and texture detail and depends on a pose estimator not trained with the policy. Maneuvers far from normal walking, such as freeing a leg stuck in a hole or climbing a high ledge, are out of reach.

Figures

From Figure 1: Deployments over multiple seasons on slippery slopes, vegetation, steep stairs, fog, underground caves... — From Figure 1: Deployments over multiple seasons on slippery slopes, vegetation, steep stairs, fog, underground caves and snow, all traversed with zero failures.

From Figure 6: A teacher trained with RL on privileged simulation data is distilled into a student that imitates its ... — From Figure 6: A teacher trained with RL on privileged simulation data is distilled into a student that imitates its actions and reconstructs the true terrain from noisy height samples; the student is deployed zero-shot on height samples from a robot-centric elevation map.

From Figure 7: The height-sample noise model and its zero, nominal, large-offset and large-noise settings (A, B), and... — From Figure 7: The height-sample noise model and its zero, nominal, large-offset and large-noise settings (A, B), and the belief encoder and decoder, whose attentional gate controls how much exteroceptive data passes through (C, D).

From Figure 4: Comparison with a proprioceptive baseline: step-climbing success versus step height (A, B), an obstacl... — From Figure 4: Comparison with a proprioceptive baseline: step-climbing success versus step height (A, B), an obstacle course the baseline only finished with human help (C-F), and forward, lateral and turning speed on flat ground and over a 20 cm step (G, H).

Abstract

Legged robots that can operate autonomously in remote and hazardous environments will greatly increase opportunities for exploration into under-explored areas. Exteroceptive perception is crucial for fast and energy-efficient locomotion: perceiving the terrain before making contact with it enables planning and adaptation of the gait ahead of time to maintain speed and stability. However, utilizing exteroceptive perception robustly for locomotion has remained a grand challenge in robotics. Snow, vegetation, and water visually appear as obstacles on which the robot cannot step~-- or are missing altogether due to high reflectance. Additionally, depth perception can degrade due to difficult lighting, dust, fog, reflective or transparent surfaces, sensor occlusion, and more. For this reason, the most robust and general solutions to legged locomotion to date rely solely on proprioception. This severely limits locomotion speed, because the robot has to physically feel out the terrain before adapting its gait accordingly. Here we present a robust and general solution to integrating exteroceptive and proprioceptive perception for legged locomotion. We leverage an attention-based recurrent encoder that integrates proprioceptive and exteroceptive input. The encoder is trained end-to-end and learns to seamlessly combine the different perception modalities without resorting to heuristics. The result is a legged locomotion controller with high robustness and speed. The controller was tested in a variety of challenging natural and urban environments over multiple seasons and completed an hour-long hike in the Alps in the time recommended for human hikers.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong