Inner Product Regularization for Imitation Learning

Benjamin A. Christie, Tianyu Qiu, Amy H. Choi, David Fridovich-Keil, Dylan P. Losey

November 2026

TL;DR InneR regularizes imitation-learning policies so that nearby states ignore the same state perturbations, by penalizing the shared null space of the policy's averaged Jacobian or Fisher information. On a real Franka arm with three distractor objects, it raises a diffusion policy's average success from 18% to 57% without extra data.

What they do

Policies learned by imitation often react to state changes that do not matter, such as a fruit moved on the table. Existing regularizers penalize the norm of the policy Jacobian (or Fisher information) at each state, which damps every response, including the relevant ones. InneR instead asks that nearby states ignore the same perturbations: if moving a distractor does not change the action at one state, it should not change it at a nearby state either.

How it works

Induced inner product. Pulling the action-space inner product back through the policy Jacobian gives an inner product on state space whose null space holds the perturbations with no first-order effect on the action. For stochastic policies this is the Fisher information with respect to state, estimated for diffusion policies from the gradient of the denoising ELBO.
Shared null space penalty. Average these matrices over a group of states and penalize the smallest eigenvalues, divided by the detached trace. The penalty is zero when the states share a null space, so it does not reward shrinking the policy’s sensitivity.
Local clusters. Each batch state is grouped with its k=8 neighbours by Grassmann distance between Jacobian row spaces, so which objects matter can change over the task. The loss is added to the behavior cloning loss.

Results

Point-mass simulation: as irrelevant dimensions grow from 8 to 32, MLP behavior cloning error rises from 0.5 to 7.4 while InneR stays at 0.2-0.9. Frobenius and nuclear norm penalties are worse than BC at every size.
Drone landing under train-test shift: 88.4% vs 77.0% (MLP) and 76.6% vs 59.4% (diffusion) success for InneR vs BC.
Real Franka, diffusion policy, 20 trials per condition: with three distractors, average success over close, stack and push is 56.7% for InneR vs 18.3% for BC. With no distractors they are similar (75.0% vs 73.3%). The nuclear-norm penalty averages 21.7% over both conditions and never succeeds at stack.

Limitations

Training is 5-6.5x slower, and the cost grows with state dimension, so policies take low-dimensional object poses from a segmentation pipeline rather than images. Under added observation noise InneR’s success falls about as fast as the baselines’, and it fails at 2 mm.

Figures

From Figure 1: A robot learning to close a coffee machine meets a test state with a task-relevant perturbation (the m... — From Figure 1: A robot learning to close a coffee machine meets a test state with a task-relevant perturbation (the machine moves, Δ1) and an irrelevant one (the banana moves, Δ2). Under behavior cloning the responses are inconsistent across nearby states, so the policy reacts to both; InneR enforces the same response to a perturbation at nearby states, so it still reacts to Δ1 but ignores Δ2.

From Figure 10: The three real-world tasks at the start, middle and end of a successful InneR rollout with three dist... — From Figure 10: The three real-world tasks at the start, middle and end of a successful InneR rollout with three distractors (apple, orange and banana): close latches the lid of a coffee maker, stack places the black cube on the tan cube, and push moves a pot into a target region.

From Figure 11: Real-world success rates of BC, Nuclear and InneR over 20 trials each: close with zero to three distr... — From Figure 11: Real-world success rates of BC, Nuclear and InneR over 20 trials each: close with zero to three distractors, and stack and push with zero and three. Averaged across the tasks, InneR matches BC with no distractors (75.0% vs. 73.3%) and beats both baselines with three.

Abstract

Robots that learn from demonstrations should extrapolate their policy despite state perturbations such as sensor noise, scene variation, and compounding errors. One approach to this robustness is regularization. Prior regularizers consider each state on its own and limit how sensitive the policy is to any perturbation at that state. By contrast, we explore how the policy's output shifts when given the same perturbation at different states. Our insight is that, locally, the robot should respond to state perturbations in a consistent manner: e.g., if changing an irrelevant object position does not impact the policy at one state, that same change should not shift the policy output at a nearby state. Applying this insight, we develop inner product regularization (InneR). We show that the inner product on action space induces an inner product on state space through the policy Jacobian (the Fisher information for stochastic policies), and that the null space of this inner product governs which state perturbations have a first-order effect on the policy. By clustering states whose inner products are aligned, averaging the inner products within each cluster, and minimizing the smallest eigenvalues of that average, InneR shapes the learned policy so that nearby states share a common null space. InneR enforces consistent responses to perturbations without limiting policy expressiveness, and can be appended to existing imitation learning pipelines for deterministic or stochastic policies. In simulated experiments, InneR makes learned policies more robust to irrelevant state dimensions and shifts between training and test distributions, whereas norm penalties often perform worse than behavior cloning. On a real robot arm performing three manipulation tasks, InneR raises the average success rate of a diffusion policy from 18% to 57% without additional data.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong