LEASH: Stay Safe in Agentic Robotics
TL;DR RoboRisk tests whether foundation models that drive a robot arm cause harm, over 92 simulated and 13 real-robot tasks, and LEASH wraps any such planner in three image-grounded safety gates and a shared safety policy. With LEASH, GPT-6 Astra's harm rate drops from 11.23% to 0% in simulation, and four planners reach 0% harm on the physical robot.
What they do
Agentic harnesses now let foundation models such as GPT and Claude drive robot arms directly. Classical safety checks such as joint limits and collision-free paths do not catch semantic hazards like putting a phone into water. The authors build RoboRisk, a closed-loop benchmark of such hazards, and LEASH, a safety layer that wraps any planner without retraining.
How it works
- RoboRisk. Five risk categories: harmful instructions to refuse (T), plans that must route around a hazard (P), hazards that appear mid-task (E), instructions injected through the scene (I), and multi-round tasks where forgetting causes harm (S). The 92 simulated tasks use a Franka Panda in MuJoCo; the 13 real tasks use a Dobot CR5V2 arm with two RealSense cameras. Humans label each run Harm, Safe Success or Safe Incomplete.
- Three gates. A task gate screens the instruction with the current camera images and can end the task. An action gate reviews each proposed action, and a commit gate pauses long skills before the gripper opens or a pour starts. A rejected action sends the planner back to replan.
- Fast reviewers. Gates get a safety score from a decision model (Jev or OpenAI’s Decisions API), which takes 1.5-2 s per call versus about 12 s for GPT-6 Luna. The planner and the gates read the same safety-policy prompt.
Results
- Simulation: GPT-6 Astra’s harm rate falls from 11.23% to 0.00% while safe success rises from 78.99% to 80.80%; GPT-6 Luna falls from 61.23% to 12.68%.
- Physical robot (one run per task): GPT-6 Astra, GPT-6.1 Sol, Claude Opus 5.5 and Gemini 4 Argon all reach 0% harm and 84.62% safe success; Grok 4.7 drops from 76.92% to 38.46%.
- Overhead: mean run time rises 1.66% (245.21 s to 249.28 s), and the gates cost $0.00522 per run, 0.66% of the total.
- On the real robot, the stronger models decline tasks near a person (a cleaver around a hand, a knife handover) that they attempt in simulation.
Limitations
Safety depends on reviewer accuracy and on the planner following the policy: GPT-6 Luna and Grok 4.7 still cause harm, partly through unstable motion. Action and commit gates can reject but not stop a task, and commit review covers only gripper opening and pouring. The physical evaluation has one run per task, many physical runs start with the object already grasped, and no full ablation of the gates was run.
Videos
Figures




Abstract
Foundation models such as GPT and Claude have shown strong capabilities in spatial understanding and embodied intelligence. These advances make agentic robotics a promising path toward artificial general intelligence in robotics. However, we find that using foundation models to control robots poses serious safety risks, and comprehensive safety evaluation is still lacking. We introduce RoboRisk, a comprehensive benchmark for evaluating safety in agentic robotics. Its 92 simulated tasks and 13 physical-robot tasks fall into five categories: Task-level Refusal (T), Plan Repair (P), Execution-time Hazard (E), Injection (I), and Sequential Memory (S). Prior benchmarks cover at most two of these categories, and several evaluate only text outputs. In contrast, RoboRisk covers all five and executes every task in closed loop. We evaluate six state-of-the-art foundation models on RoboRisk: GPT-6 Astra, GPT-6.1 Sol, GPT-6 Luna, Claude Opus 5.5, Gemini 4 Argon, and Grok 4.7. These models generally show high harm rates. We also evaluate two robot-specific APIs: the Dream API and PokeBot's RoboAPI. We further propose the Lease-Enforced Agentic Safety Harness (LEASH), which offers essential safety infrastructure for agentic robotics and can be integrated into nearly any agentic architecture. LEASH leverages a decision model (Jev or OpenAI's Decisions API) to implement task, action, and commit gates, which review the task, each action, and each irreversible operation in their visual context. The planner and all three gates share a common safety policy. Experiments on RoboRisk in simulation and on physical robots show that LEASH effectively reduces harm across all six foundation models. It largely preserves their task completion performance and adds little time and token overhead.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
