← Paper Digest

Distilling Safety Filters for Filter-free Multi-agent RL

Srikar Babu Gadipudi, Somayeh Sojoudi, Claire J. Tomlin

November 2026

Crazyflie quadrotors cross between stacked-bucket obstacles with no safety filter; inset shows the top-down view.

TL;DR Train multi-robot navigation with a centralized CBF safety filter, then distill the filter into a decentralized policy that runs without it. With no filter and only local sensing, the policies had no collisions in 1,800 simulated episodes and flew up to five Crazyflie quadrotors.

What they do

CBF safety filters keep multi-agent RL collision-free during training, but for a team the filter is a centralized QP whose constraints grow quadratically with team size: 16.9 ms for 20 agents, a third of the 50 ms control period. The authors train with the filter and deploy a decentralized policy without it, using only local sensing and no communication.

How it works

  • Learn only from executed actions. The filter maps many raw actions to the same safe action, so actor-critic methods such as SAC overvalue unsafe proposals. Implicit Q-learning instead regresses the policy onto the executed safe actions, weighted by advantage.
  • Off-support safety-filter distillation (OSFD). Filtered rollouts never reach near-collision states. OSFD samples synthetic states with agents close to obstacles, walls or each other and moving toward them, labels them with the training filter and trains the actor on those labels, with no environment interaction.
  • Decentralized actor. Attention over one token per sensed agent or obstacle (barrier value, normal, relative velocity), so its parameters do not depend on team size.
  • Guarantee. The distillation loss bounds the policy’s constraint violation, violations do not accumulate over time, and if the distillation states cover the deployed states the loss also bounds the collision probability.

Results

  • Local sensing, no filter: 0 collisions in 1,800 episodes. CBF-RL collides in 23-67% of double-integrator episodes with three agents.
  • Trained with 3 agents, run zero-shot with up to 20: no collisions up to 8 agents, 3.3% of random-layout episodes at 20. The policy takes 0.6 ms per agent at every team size.
  • Without OSFD, SAC collides in 98.3% of episodes and IQL in 8.0%.
  • Hardware: in 17 of 18 filter-free flights with 3 or 5 Crazyflies, every drone reached its goal without contact.

Limitations

Disc-shaped agents and circular obstacles in a plane, with integrator dynamics. The collision bound is proved for single integrators; for double integrators a discretization term can use most of the margin, so safety there is only measured. In the hardest scene the local policy reaches 93-95% of goals. Flights used motion capture and an offboard laptop, and in one tight layout two drones touched.

Videos

Another filter-free flight through a different obstacle layout.
Simulation with a large team: IQL alone collides, IQL with filter distillation does not.

Figures

From Figure 1: (a) Training and deployment. During training a centralized CBF-QP filter corrects the agents' actions;...
From Figure 1: (a) Training and deployment. During training a centralized CBF-QP filter corrects the agents' actions; the policy is updated with IQL on the executed safe actions and with filter distillation at sampled near-collision states. At deployment each agent acts on local sensing without the filter, and the policy scales zero-shot to more agents.
From Figure 1: (b) Trajectories of Crazyflie quadrotors flying the policy among stacked-bucket obstacles, with no fil...
From Figure 1: (b) Trajectories of Crazyflie quadrotors flying the policy among stacked-bucket obstacles, with no filter.
From Figure 3: Zero-shot scaling of the 3-agent double-integrator policies (local sensing) to up to 20 agents with 2 ...
From Figure 3: Zero-shot scaling of the 3-agent double-integrator policies (local sensing) to up to 20 agents with 2 obstacles, without the filter, in simulation. (a) Collisions, (b) goals reached, (c) per-step compute of one agent's policy versus the joint CBF-QP.
Abstract

Reinforcement learning (RL) can teach robot teams to navigate shared spaces, but trial and error causes collisions that real robots cannot afford. Control barrier function (CBF) safety filters prevent these collisions by correcting unsafe actions, yet in a team the filter becomes a centralized optimization whose number of constraints grows quadratically with the team size. We propose an approach that builds the filter's safety into the RL policy and examine how such a system behaves during filter-free deployment. However, the filter that keeps training safe also hides what the policy must learn: it masks the cost of unsafe actions, which actor-critic methods then overvalue, and it keeps near-collision states out of the training data. Our approach therefore learns only from executed safe actions with implicit Q-learning and distills the filter into the policy at synthetic near-collision states. We prove that filter distillation loss bounds the policy's constraint violation and hence how far agents can cross their safety margin, so errors never accumulate. When distillation states cover deployed states, the loss also bounds the collision probability at every step. Without a filter and acting only on local observations, our policies have no collisions in 1,800 test episodes with multiple agents. A policy trained with three agents transfers zero-shot to teams of twenty at constant per-agent computation and stays collision-free with up to eight agents. We test our policies on Crazyflie quadrotors with up to five flying at the same time.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.