Matt Strong

Matt Strong

CS PhD at Stanford University

← Paper Digest

Caging Vision-Language Navigation for Legged Robots via Barrier-Certified Safety Architecture

Abhijeet M. Kulkarni, Saimouli Katragadda, Yuze Du, Zijun Sha, Guoquan Huang

November 2026

TL;DR A safety layer sits between a VLM/VLA navigation planner and a legged robot. It turns the model's actions into goal poses and certifies every command with a sampling-based MPPI planner under barrier constraints on a live signed distance field. On long missions with a Unitree G1, collisions dropped from 60 of 60 runs to 0 of 60 across four language front ends.

What they do

Vision-language models can turn spoken instructions into navigation commands for legged robots, but those commands ignore the robot’s footprint and the obstacles around it. This paper “cages” the model: its outputs are treated only as proposals, and a geometric planning layer checks every motion for collisions before the robot executes it. The same layer runs on a Unitree G1 humanoid and a Ghost Robotics Vision 60 quadruped.

How it works

  • Semantic layer. A VLM picks discrete actions (turns of up to 45°, steps of up to 75 cm). A goal projector moves each resulting waypoint to a free spot with at least 0.3 m clearance. An optional Grounding-DINO + SAM2 stage then approaches a named object.
  • Physical layer. An incrementally updated signed distance field (SDF) of the surroundings. The robot is modeled as a few spheres, and a Bézier-parameterized MPPI planner samples 200 trajectories over a 2.5 s horizon.
  • Certification. Discrete exponential barrier constraints limit how fast clearance may shrink between steps. Only a rollout that satisfies them is executed, otherwise the robot stops. Under the stated assumptions this provably keeps it collision-free.

Results

  • Controlled G1 ablations across ChatGPT, Claude, Gemini and NaVILA: collision rate falls from 80% to 0% without a global planner and from 100% to 0% with one. End-to-end success rises from 15% to 40% and from 0% to 70%.
  • G1, 75 trials per configuration: collision rate falls from 0.33 to 0 on simple tasks and from 0.50 to 0 on complex ones, with success unchanged at 0.75. On the Vision 60 it falls from 0.67 to 0.20 and from 0.91 to 0.08.
  • Long G1 missions: 60/60 uncaged runs collide, 0/60 caged.
  • Object grounding cuts Vision 60 endpoint error from 1.50 m to 0.29 m.

Limitations

The guarantee only covers geometry already in the map. Thin wires, table edges and LiDAR blind spots caused the remaining Vision 60 collisions. The model ignores state-estimation error, latency and tracking error, and only static obstacles are handled. The cage cannot fix wrong semantic goals.

Figures

From Figure 1: The Legged-VLN architecture separates semantic grounding (top), where a VLA/VLM turns the instruction ...
From Figure 1: The Legged-VLN architecture separates semantic grounding (top), where a VLA/VLM turns the instruction into 2D goals, from safety-assured physical grounding (bottom), where state estimation, local 3D perception and dEBF-constrained MPPI produce velocity commands.
From Figure 3: Qualitative effects of dEBF in simulation. (a) Larger gamma keeps more clearance from an obstacle. (b)...
From Figure 3: Qualitative effects of dEBF in simulation. (a) Larger gamma keeps more clearance from an obstacle. (b) Speed in a narrow passage for collision-penalty MPPI and MPPI+dEBF. (c) Smooth slowdown during a final object approach.
From Figure 4: Unitree G1 following
From Figure 4: Unitree G1 following "Move forward, step onto the black mat, and stop at the TV monitor in front of the window on the right."
From Figure 4: Ghost Robotics Vision 60 following
From Figure 4: Ghost Robotics Vision 60 following "Go out the door then turn left and stop at the bike."
Abstract

By harnessing the inherent commonsense capabilities of pretrained Large Language Models (LLMs) and Vision-Language Models (VLMs), current Vision-Language Action(VLA) models for legged robots excel at interpreting language-based navigation commands but lack the physical grounding and explicit safety assurance required for real-world operation. Therefore, this paper aims to cage Legged-Vision-Language Navigation (VLN) through a safety architecture that provides physical grounding and ensures navigation safety. The key idea is not to abandon the powerful capabilities of end-to-end VLA planning, but rather to augment and cage them. In particular, we integrate a high-level VLA planner with a low-level physical-grounding layer that leverages efficient state estimation, in-situ 3D spatial perception, and an Signed Distance Field (SDF) representation to generate executable trajectories. We provide a navigation planner with provable collision-avoidance guarantees under the stated assumptions by combining a low-dimensional Bezier-parameterized MPPI planner with Discrete Exponential Barrier Function (dEBF) constraints. The proposed Legged-VLN system is modular, making it compatible with various VLM/VLA models and agnostic to the robot platform. The proposed system is validated with extensive real-world experiments on humanoid and quadrupedal robots, demonstrating versatile, efficient, and safe vision-language driven navigation capabilities.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.