PROOF: Perceptive Robustness via Out-of-Distribution Fallback for Legged Locomotion
TL;DR A depth-camera parkour policy for the Unitree Go2 that estimates how uncertain its terrain features are and switches to proprioception-only walking when vision is corrupted. Under strong simulated depth corruption it keeps 95-100% traversal success where a student without a learned fallback drops to 0%.
What they do
Depth-based parkour policies fail when the camera is occluded, overexposed or sees reflections, because the corrupted image drives the actions. PROOF trains a quadruped parkour policy that detects out-of-distribution depth, drops it, and keeps walking on proprioception alone, on a Unitree Go2 with one RealSense D435.
How it works
- Teacher. An elevation-map policy trained with PPO in Isaac Lab, with relaxed forward velocity tracking so it can build momentum before jumps. Its terrain latent is randomly masked, so it learns both parkour and blind fallback walking.
- Student. A CNN on two 10 Hz depth frames plus a GRU predicts the teacher’s terrain latent and its log-variance, the latter trained with a negative log-likelihood loss as a reliability signal.
- Gate. A head trained on injected depth corruptions zeroes the terrain latent, using a hard mask with hysteresis (opens at 0.8, closes below 0.5). During corruption the student imitates the masked teacher.
- Training takes about 4.5 h on one RTX 4090; the policy runs onboard on a Jetson Nano.
Results
- Simulation with strong depth corruption (100 episodes per terrain): 95-100% success on five non-parkour terrains, against 0% without teacher masking and 73-96% with plain MSE distillation and no gate.
- As a corruption detector the gate reaches AUROC 0.999 on seen and 0.995 on unseen corruptions.
- Hardware corruptions (occlusion, flashlight, mirror, blur, dropout, noise; 5 trials at each of three speeds): every flat-ground trial succeeds. On stairs, motion blur drops to 0.6 and 0.4 at 0.5 and 1.0 m/s.
- Real parkour (10 trials per setting): highest success on the widest gap tested.
Limitations
The corruptions are hand-designed, and the log-variance is a relative signal, not a calibrated error. Under severe corruption the robot only walks blind, since parkour needs vision.
Videos
Figures



Abstract
Vision-based perceptive locomotion has enabled legged robots to achieve agile and terrain-aware locomotion across complex environments. However, the reliability of such systems heavily depends on visual observations, which are inherently susceptible to occlusion, reflective surfaces, and illumination variations, often resulting in catastrophic locomotion failures under degraded perception conditions. In this paper, we present PROOF (Perceptive Robustness via Out-of-Distribution Fallback for Legged Locomotion), a perceptive locomotion framework that enables robust locomotion under severe visual disturbances. Our method adopts a two-stage teacher-student learning paradigm that explicitly models perception reliability: the teacher learns both perception-driven and proprioceptive fallback behaviors, while the student predicts exteroceptive features with uncertainty and suppresses them under out-of-distribution inputs. We evaluate our method on a Unitree Go2 quadruped equipped with a RealSense D435 depth camera in real-world settings. Under normal visual conditions, our method enables the robot to successfully perform high-dynamic parkour maneuvers, including traversing large gaps, elevated platforms, and narrow beam terrains. Under severe visual degradation, such as heavy occlusion and strong illumination changes, the robot continues to maintain stable and robust locomotion performance.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
