Legged Locomotion in Challenging Terrains using Egocentric Vision

Ananye Agarwal, Ashish Kumar, Jitendra Malik, Deepak Pathak

Seminal November 2022

Stepping from a table onto a row of stools used as stepping stones.

TL;DR A small quadruped crosses stairs, curbs, gaps and stepping stones with one front-facing depth camera and a policy trained in simulation that maps depth straight to joint angles, with no elevation map. On the real robot it succeeds 100% on stairs and 26 cm gaps and 94% on stepping stones, where a blind policy gets 0%.

What they do

Vision-based legged locomotion usually fuses depth images into a metric elevation map, which needs odometry and comes out noisy. Here a Unitree A1 with one front-facing RealSense camera maps egocentric depth and proprioception directly to target joint angles. The camera cannot see the ground under the hind feet, so a recurrent network keeps a short memory of recent views. No gait priors are imposed: the robot’s hip is only 28 cm high, and gaits such as a hip-abduction move for climbing curbs emerge from RL.

How it works

Phase 1, RL. PPO in IsaacGym on scandots (terrain heights at fixed points around the robot) as a cheap stand-in for depth, plus privileged friction, mass and motor strength. Rewards penalize energy and collisions; a curriculum runs over stairs, slopes, gaps, stepping stones and obstacles. 15 billion samples, about 13 h.
Phase 2, distillation. DAgger trains a student that sees only depth and proprioception to match the phase-1 actions: 6 million samples, about 6 h.
Two architectures. A monolithic GRU, or an RMA variant where a GRU estimates the terrain latent from depth and feeds the frozen phase-1 MLP policy.
Deployment. 50 Hz on the onboard Jetson NX and UPboard, with no mapping or localization.

Results

Simulation: the two variants travel 134 m and 136 m in total, versus 73 m for a baseline distilled onto a noisy elevation map and 85 m for a blind one. On stepping stones they cover 19-21 m, the baselines about 1 m.
Real robot: 100% success up and down 13-stair flights and across 26 cm gaps, 94% on stepping stones. The blind policy completes no upstairs run (2.2 stairs on average) and no gap or stepping-stone trial; it does descend stairs, with a falling gait that detached the robot’s rear right hip.
Climbs stairs up to 24 cm and curbs up to 26 cm, and walked outdoor trails with dirt, pebbles and rocks.

Limitations

Failures come from visual or terrain mismatch between simulation and reality, and the only fix is to recreate the case in simulation and retrain. The real-world tests used separate phase-2 policies and camera positions for stairs and for gaps and stepping stones. The robot sometimes misses a step and recovers, and outdoors it sometimes gets stuck in crevices.

Videos

Walking down an indoor staircase.

Walking up rough stone steps on a wooded trail.

Figures

From Figure 1: The robot crossing stepping stones, curbs, stairs and rocky terrain indoors and outdoors, day and nigh... — From Figure 1: The robot crossing stepping stones, curbs, stairs and rocky terrain indoors and outdoors, day and night, using a single front-facing depth camera.

From Figure 3: Two-phase training. Phase 1 trains a policy with RL on cheap-to-compute scandots; phase 2 trains a stu... — From Figure 3: Two-phase training. Phase 1 trains a policy with RL on cheap-to-compute scandots; phase 2 trains a student that sees only the front depth image and proprioception to imitate it. Shown for both architectures: RMA, where estimated vision and proprioception latents condition a feedforward base policy, and a monolithic GRU.

From Figure 4: Real-world success rates of the vision policy and a blind baseline on upstairs and downstairs (17 cm h... — From Figure 4: Real-world success rates of the vision policy and a blind baseline on upstairs and downstairs (17 cm high, 30 cm deep steps), stepping stones (30 cm wide, 15 cm apart) and gaps (26 cm apart), with the number of stairs or stones crossed.

Abstract

Animals are capable of precise and agile locomotion using vision. Replicating this ability has been a long-standing goal in robotics. The traditional approach has been to decompose this problem into elevation mapping and foothold planning phases. The elevation mapping, however, is susceptible to failure and large noise artifacts, requires specialized hardware, and is biologically implausible. In this paper, we present the first end-to-end locomotion system capable of traversing stairs, curbs, stepping stones, and gaps. We show this result on a medium-sized quadruped robot using a single front-facing depth camera. The small size of the robot necessitates discovering specialized gait patterns not seen elsewhere. The egocentric camera requires the policy to remember past information to estimate the terrain under its hind feet. We train our policy in simulation. Training has two phases - first, we train a policy using reinforcement learning with a cheap-to-compute variant of depth image and then in phase 2 distill it into the final policy that uses depth using supervised learning. The resulting policy transfers to the real world and is able to run in real-time on the limited compute of the robot. It can traverse a large variety of terrain while being robust to perturbations like pushes, slippery surfaces, and rocky terrain. Videos are at https://vision-locomotion.github.io

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong