The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systems

Kai-Chieh Hsu, Haimin Hu, Jaime Fernández Fisac

Seminal September 2023

TL;DR A review that casts HJ reachability, control barrier functions, model predictive shielding and learned safety critics as one structure, a fallback policy plus a safety monitor plus an intervention scheme. One Universal Safety Filter Theorem then rederives the safety guarantee of each class.

What they do

A review of safety filters: processes that monitor an autonomous system at runtime and override its intended control only when it could lead to a catastrophic failure. The authors argue that seemingly different methods share one modular structure, organize them into a taxonomy, and prove a single theorem from which each class’s guarantee follows.

How it works

Safety-performance separation. A least-restrictive filter overrides only actions that would leave the maximal safe set, so a task policy (or an RL learner) can be designed without regard for safety.
Three parts. Every filter has a fallback policy, a safety monitor (value-based, such as an HJ value function or CBF, or rollout-based, forward-simulating the fallback) and an intervention scheme (a switch to the fallback, or an optimization such as the CBF-QP).
Universal Safety Filter Theorem. If the monitor is non-negative only when the fallback can keep the system safe after the action, and intervention blocks failing actions whenever the fallback itself passes, then from any initial state the monitor deems safe the system stays safe under any task policy.

Results

Corollaries rederive the guarantees of the HJ switching filter, the CBF-QP filter, rollout filters with forward-reachable sets, and a braking filter for navigating unmapped space.
Trade-offs: grid-based HJ cost is exponential in state dimension, limiting it to about six continuous state variables; CBFs intervene smoothly but lack general synthesis methods; learned safety critics scale but carry no guarantees without statistical assumptions.
Suggested combinations: HJ value functions used as CBFs, and RL-trained fallbacks certified at runtime by robust rollouts. Hardware examples include a kicked quadruped, a quadrotor in wind and interactive driving.

Limitations

The unified theory is developed for robust filters; probabilistic counterparts are stated to be analogous but not derived. Most of the exposition assumes perfect state observation. Open problems the authors list: filters that account for runtime learning, for interaction with other agents, and for high-dimensional sensory inputs and learned representations.

Figures

From Figure 1: Examples of safety-critical systems: a robot vacuum on a grid world, an autonomous car on a highway, a... — From Figure 1: Examples of safety-critical systems: a robot vacuum on a grid world, an autonomous car on a highway, and a quadruped walking indoors.

From Figure 2: A safety filter has a safety monitor that checks whether the task policy's proposed control is safe, a... — From Figure 2: A safety filter has a safety monitor that checks whether the task policy's proposed control is safe, an intervention scheme that modifies it as needed, and a fallback policy that informs both. Monitoring is value-based or rollout-based; intervention either switches to the fallback or optimizes the control within the confines of safety.

From Figure 3: The reviewed safety filters, arranged by type of guarantee versus how the fallback strategy is synthes... — From Figure 3: The reviewed safety filters, arranged by type of guarantee versus how the fallback strategy is synthesized (left), and by monitoring versus intervention scheme (right).

From Figure 4: Safety filters on hardware. (a) A quadruped uses an adversarial-RL safety filter to avoid falling when... — From Figure 4: Safety filters on hardware. (a) A quadruped uses an adversarial-RL safety filter to avoid falling when kicked. (b) A quadrotor learns to hover in wind with an HJ fallback policy. (c) Shielding-aware planning with active uncertainty reduction gives a safe, chatter-free driving trajectory. (d) An MPC planner aware of an HJ filter; the filter intervenes (star) when the safety value approaches 0.

Abstract

Recent years have seen significant progress in the realm of robot autonomy, accompanied by the expanding reach of robotic technologies. However, the emergence of new deployment domains brings unprecedented challenges in ensuring safe operation of these systems, which remains as crucial as ever. While traditional model-based safe control methods struggle with generalizability and scalability, emerging data-driven approaches tend to lack well-understood guarantees, which can result in unpredictable catastrophic failures. Successful deployment of the next generation of autonomous robots will require integrating the strengths of both paradigms. This article provides a review of safety filter approaches, highlighting important connections between existing techniques and proposing a unified technical framework to understand, compare, and combine them. The new unified view exposes a shared modular structure across a range of seemingly disparate safety filter classes and naturally suggests directions for future progress towards more scalable synthesis, robust monitoring, and efficient intervention.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong