Legged locomotion
Quadrupeds and bipeds that walk, run and climb, mostly with controllers learned in simulation.
Start here: seminal papers
Passive Dynamic WalkingSeminal
The International Journal of Robotics Research, April 1990
Two stiff legs with curved feet, pin-jointed at the hip, settle into a steady human-like gait down a shallow slope with no active control or energy input besides gravity. A step-to-step analysis predicts the gait and its stability, and a 50 cm test biped walks at 0.46 m/s on a 2.5% slope.
Templates and Anchors: Neuromechanical Hypotheses of Legged Locomotion on LandSeminal
Journal of Experimental Biology, December 1999
A perspective that splits locomotion models into templates, the simplest models that capture a behavior (such as the spring-loaded inverted pendulum for running), and anchors, detailed models of legs, joints and muscles whose control collapses them onto a template. It offers twelve hypotheses, not new experiments.
Hybrid Zero Dynamics of Planar Biped WalkersSeminal
IEEE Transactions on Automatic Control, January 2003
A feedback design for underactuated planar bipeds that keeps a two-dimensional zero-dynamics surface invariant through foot impacts, so walking stability reduces to a scalar linear return map. On a 5-link biped model, optimized controller parameters give exponentially stable walking at 1.05 m/s with a peak torque of about 47 Nm.
Efficient Bipedal Robots Based on Passive-Dynamic WalkersSeminal
Science, February 2005
Three bipeds built at Cornell, Delft and MIT add small ankle or hip actuators to passive-dynamic ramp walkers, so they walk on level ground with very simple control. The Cornell biped's mechanical cost of transport is about 0.055, close to a human's 0.05; the authors estimate 1.6 for Honda's Asimo.
Compliant Leg Behaviour Explains Basic Dynamics of Walking and RunningSeminal
Proceedings of the Royal Society B, November 2006
Add a second spring leg to the spring-mass running model and it walks, reproducing the M-shaped vertical ground reaction force, small body bob and out-of-phase energy changes of human walking at about 1.2 m/s, which the stiff-legged inverted pendulum cannot. The same model also runs, above a speed gap of about 1.5 m/s.
Emergence of Locomotion Behaviours in Rich EnvironmentsSeminal
July 2017
They train simulated walker, quadruped and humanoid bodies with a distributed PPO on procedurally generated obstacle courses, rewarding mostly forward velocity. Running, jumping, crouching and turning emerge without behaviour-specific reward shaping, and the Planar Walker learns to clear hurdles nearly as tall as its own body.
Feedback Control of an Exoskeleton for Paraplegics: Toward Robustly Stable Hands-free Dynamic WalkingSeminal
February 2018
The authors port hybrid-zero-dynamics control from bipedal robots to the 12-actuator ATALANTE exoskeleton so complete paraplegics walk without crutches. Three patients walked about 9-10.5 m hands-free at 0.11-0.15 m/s, and a newer learned controller recovers from 750 N pushes in simulation where a fixed gait fails above 300 N.
Sim-to-Real: Learning Agile Locomotion For Quadruped RobotsSeminal
April 2018
Trains Minitaur quadruped gaits with PPO in PyBullet and narrows the sim-to-real gap with a measured actuator model, simulated latency, randomization and a compact observation space. The learned gallop and trot run on the real robot without fine-tuning, as fast as expert-tuned gaits with 35% and 23% less power.
Learning to Walk via Deep Reinforcement LearningSeminal
December 2018
Soft actor-critic with its temperature tuned automatically toward a target entropy, used to teach a Minitaur quadruped to walk from scratch on the real robot. A gait emerges in about two hours (about 400 rollouts) and handles a slope, wooden blocks and steps it never trained on.
Learning agile and dynamic motor skills for legged robots★ Must read
January 2019
Train ANYmal's controllers with RL in a simulator that uses a neural network, fitted to real robot data, to model the robot's series-elastic actuators, then deploy the policies directly on the robot. The learned locomotion policy tracks commands more accurately and with less torque than the best model-based controller, reaches 1.5 m/s (25% above the previous record), and gets the robot up after a fall.
Crocoddyl: An Efficient and Versatile Framework for Multi-Contact Optimal ControlSeminal
September 2019
An open-source optimal control library for legged robots with a given contact sequence, built around FDDP, a DDP variant that starts from infeasible guesses and keeps dynamics gaps open in early iterations. Gaits, jumps and a front-flip converge in 10-34 iterations and under 0.5 s, with single iterations running at up to 859.6 Hz.
Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive ControlSeminal
September 2019
A convex MPC plans ground reaction forces on a simple lumped-mass model, and a 500 Hz whole-body impulse controller tracks those forces, rather than a body trajectory, using full-body dynamics. On the Mini-Cheetah it runs five gaits outdoors with one set of gains and reaches 3.7 m/s on a treadmill, up from 2.45 m/s with MPC alone.
Learning Agile Robotic Locomotion Skills by Imitating AnimalsSeminal
April 2020
A quadruped learns gaits, spins and hops in simulation by imitating retargeted dog mocap and animator clips, then adapts on the real robot by searching a learned latent dynamics space in about 50 trials. The learned dog trot reaches 1.08 m/s, faster than the manufacturer's 0.84 m/s gait.
Learning Quadrupedal Locomotion over Challenging Terrain★ Must read
October 2020
A blind quadruped controller trained in simulation, where an RL teacher with privileged terrain information is distilled into a student that reads 2 s of joint and IMU history. It transfers zero-shot to mud, snow, vegetation and running water, and ran four 60-minute DARPA SubT missions with zero failures.
Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward CompositionSeminal
November 2020
Writing gaits as periodic costs on foot forces and foot speeds gives one reward function, with no reference motions, that trains walking, running, galloping, hopping and skipping on the Cassie biped. The policies transfer from simulation to hardware, and a single policy moves between all two-beat gaits and standing.
Robotic Guide Dog: Leading a Human with Leash-Guided Hybrid Physical InteractionSeminal
March 2021
A Mini Cheetah quadruped guides a blindfolded person on a leash, and its planner lets the leash go slack so the pair can fit through spaces a taut 1.6 m robot-human system cannot. Three adults were guided through doorways and a corridor as narrow as 1.0 m without any collision.
Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement LearningSeminal
May 2021
An LSTM policy trained with PPO in simulation on randomized staircases lets the Cassie biped climb and descend real stairs blind, using only proprioception and the existing flat-ground reward. On an outdoor staircase it succeeded in 8 of 10 ascents and 10 of 10 descents.
RMA: Rapid Motor Adaptation for Legged Robots★ Must read
July 2021
RMA trains a quadruped walking policy and an online adaptation module entirely in simulation; the module infers a latent environment code from the last 0.5 s of joint states and actions, so a Unitree A1 adapts in fractions of a second with no real-world fine-tuning. It carries 12 kg, its own body weight, and reaches 73.5% success in simulation versus 62.4% for domain randomization.
Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning★ Must read
September 2021
Run simulation and PPO training for thousands of quadrupeds in parallel on a single GPU, with a game-inspired terrain curriculum. A rough-terrain ANYmal policy trains in under 20 minutes (flat terrain in under 4) and transfers to the real robot.
Learning robust perceptive locomotion for quadrupedal robots in the wildSeminal
January 2022
A simulation-trained ANYmal controller fuses noisy elevation-map height samples with proprioception through a gated recurrent encoder, so it uses the map when it is reliable and falls back on proprioception when it is not. Deployed zero-shot, it finished a 2.2 km Alpine hike in 78 minutes without a fall, close to the 76 minutes a hiking planner gives for humans.
Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged LocomotionSeminal
February 2022
Train a quadruped's locomotion policy and a neural state estimator together in one PPO loop, with the estimator supplying base velocity, foot height and contact probability to the policy. On a real Mini Cheetah it runs at up to 3.75 m/s on flat ground and 3.54 m/s on a slippery plate with friction 0.22.
Rapid Locomotion via Reinforcement LearningSeminal
May 2022
One RL policy trained in simulation, with an adaptive curriculum over velocity commands and teacher-student online system identification, runs the MIT Mini Cheetah at up to 3.9 m/s and spins it at 5.7 rad/s, deployed zero-shot.
A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement LearningSeminal
August 2022
An A1 quadruped learns to walk from scratch in the real world with model-free RL, using SAC with a regularized critic, many critic updates per step and a fast JAX implementation. It learns a gait in about 20 minutes on each of five indoor and outdoor terrains.
Perceptive Locomotion through Nonlinear Model Predictive ControlSeminal
August 2022
A nonlinear MPC for ANYmal that optimizes torso and all leg joints together, with the terrain given as convex foothold regions and a signed distance field for collisions. It crosses gaps, stepping stones and stairs on hardware with a mean MPC iteration of 6.6 ms.
Deep Whole-Body Control: Learning a Unified Policy for Manipulation and LocomotionSeminal
October 2022
One RL policy controls both the legs and the arm of a low-cost quadruped with an arm mounted on top, so the legs bend and stretch to extend the arm's reach. In real-world cup picking it succeeds 80% of the time on easy and hard points, versus 30% and 10% for a built-in MPC plus arm IK baseline.
Legged Locomotion in Challenging Terrains using Egocentric VisionSeminal
November 2022
A small quadruped crosses stairs, curbs, gaps and stepping stones with one front-facing depth camera and a policy trained in simulation that maps depth straight to joint angles, with no elevation map. On the real robot it succeeds 100% on stairs and 26 cm gaps and 94% on stepping stones, where a blind policy gets 0%.
Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of BehaviorSeminal
December 2022
Train one quadruped policy on flat ground only that can walk in many styles (gait, step frequency, body height and pitch, stance width, footswing height), so a human operator can retune it on the spot for stairs, curbs, a 22 cm bar or shoves instead of retraining. On unseen simulated platform terrain, trotting survives 0.88 of the episode vs 0.83 for a gait-free baseline.
Real-World Humanoid Locomotion with Reinforcement Learning★ Must read
March 2023
A causal transformer that reads the last 16 steps of proprioception and actions is trained with RL in simulation and deployed zero-shot on the full-sized Digit humanoid. Over one week of full-day outdoor testing it never fell, and it recovers from tripping on steps it never saw in training.
ANYmal Parkour: Learning Agile Navigation for Quadrupedal RobotsSeminal
June 2023
A navigation policy picks among five learned locomotion skills (walk, jump, climb up, climb down, crouch) and steers them using a learned 3D reconstruction of the terrain, so ANYmal D can cross parkour-style box courses. The pipeline is trained only in simulation, reaches up to 2 m/s on the real robot, and succeeds in 96-98% of simulated courses.
Robot Parkour LearningSeminal
September 2023
Five parkour skills are trained in simulation with RL that first lets the robot pass through obstacles and then enforces real contact, then distilled into one depth-camera policy for low-cost quadrupeds. On a real A1 it climbs 0.40 m obstacles and leaps 0.60 m gaps with 80% success.
Extreme Parkour with Legged RobotsSeminal
September 2023
One neural network maps depth from a single front camera on a low-cost Unitree A1 straight to joint commands and picks its own heading. Trained with RL in simulation, the robot high-jumps 2x its height, long-jumps 2x its length, crosses 37° tilted ramps and walks in a handstand.
DTC: Deep Tracking ControlSeminal
September 2023
DTC trains an RL policy in simulation to track footholds from a trajectory optimizer, giving the ANYmal quadruped model-based foot-placement accuracy together with RL robustness. It tracks footholds to 2.3 cm and crosses gaps, beams and stepping stones where an RL baseline fails.
Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion ControlSeminal
January 2024
One RL recipe trains walking, running and jumping controllers for the Cassie biped, using a policy that reads both a 2-second and a 4-step history of the robot's inputs and outputs, trained across many commanded tasks. Transferred zero-shot, Cassie ran 400 m in 2 min 34 s, 100 m in 27.06 s, and jumped 1.4 m forward and onto a 0.44 m platform.
Humanoid Locomotion as Next Token PredictionSeminal
February 2024
Train a small causal transformer to predict the next observation and action of humanoid walking trajectories, masking actions where the data has none (motion capture, YouTube videos). Trained offline on 27 hours of walking data, it walks a full-sized Digit humanoid around San Francisco zero-shot and tracks commands better than the RL policy that generated its labeled data.
Humanoid Parkour LearningSeminal
June 2024
One depth-vision RL policy lets a Unitree H1 humanoid jump, leap, climb stairs and clear hurdles, trained on fractal-noise terrain instead of motion references or foot-raising rewards. On the real robot it jumps onto 0.42 m platforms, leaps 0.8 m gaps and runs at 1.8 m/s.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.