Matt Strong

Matt Strong

CS PhD at Stanford University

← Paper Digest

Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving

Grady Williams, Paul Drews, Brian Goldfain, James M. Rehg, Evangelos A. Theodorou

★ Must read July 2017

TL;DR A sampling-based MPC derived from information theory averages thousands of GPU-simulated control sequences, weighted by their exponentiated cost, with no gradients and arbitrary nonlinear dynamics. On a 1:5 scale rally car on a dirt track it completed all 100 counter-clockwise laps at an 11 m/s target with a top speed of 9.06 m/s; a cross-entropy MPC baseline completed 66% of its laps at that setting.

What they do

The paper derives a sampling-based model predictive controller (IT-MPC) from information theory and uses it to race a 1:5 scale AutoRally car around a dirt track. The free energy gives a lower bound on the optimal control cost, and an optimal control distribution attains it. Control then means pushing the applied control distribution toward that optimum. Unlike path integral control, this does not require control-affine dynamics.

How it works

  • Update law. Minimizing the KL divergence to the optimal distribution makes the best control sequence its expected value. Importance sampling estimates it by averaging sampled control perturbations, each weighted by its exponentiated negative cost.
  • Practical fixes. Costs are shifted so the best sample costs 0, and control limits are clamped inside the dynamics. A Savitzky-Golay filter smooths the output. Under 1% of samples are zero-mean noise, so the controller can recover from strong disturbances.
  • Real time. It runs at 40 Hz with a 2 s horizon, entirely on the car’s GTX-750ti GPU.
  • Models and cost. A 25-term basis-function regression and a 1412-parameter neural network, both fit to 30 minutes of human driving. The cost combines a track map plus an impulse penalty for leaving the track, a speed target and a side-slip penalty.

Results

  • Of 24 settings, 17 reached 100 laps: over 1700 laps and more than 100 km of driving.
  • IT-MPC with the neural network succeeded on 100% of laps everywhere except 11 m/s clockwise (76%). At 11 m/s counter-clockwise it lapped in 9.27 s with a top speed of 9.06 m/s.
  • The cross-entropy baseline (CEM-MPC) was faster but less reliable. It completed 66.32% of laps at 11 m/s counter-clockwise, and with the basis-function model it hit a barrier on over 50% of trials at 8.5 m/s.
  • At high speed the controller slides slightly into turns and counter-steers out of them.

Limitations

IT-MPC is conservative: at the 8.5 m/s target its top speed stays about 1 m/s below the target. All of its failures with the neural network came at 11 m/s clockwise, where the model predicted over-steer, the car under-steered and it hit the barrier. Neither model captures hidden variables like track condition or battery voltage. Highly unstable systems are hard to sample useful trajectories from.

Figures

From Figure 2: The control objective pushes the controlled distribution as close as possible to the optimal distribut...
From Figure 2: The control objective pushes the controlled distribution as close as possible to the optimal distribution by minimizing the KL divergence between them.
From Figure 7: Trajectory traces of IT-MPC with the neural network model at the 11 m/s target, colored by speed. 100 ...
From Figure 7: Trajectory traces of IT-MPC with the neural network model at the 11 m/s target, colored by speed. 100 counter-clockwise laps, about 6 km of driving.
From Figure 8: Time-lapse of the vehicle cornering. The front left wheel is off the ground as it enters the turn.
From Figure 8: Time-lapse of the vehicle cornering. The front left wheel is off the ground as it enters the turn.
Abstract

We present an information theoretic approach to stochastic optimal control problems that can be used to derive general sampling based optimization schemes. This new mathematical method is used to develop a sampling based model predictive control algorithm. We apply this information theoretic model predictive control (IT-MPC) scheme to the task of aggressive autonomous driving around a dirt test track, and compare its performance to a model predictive control version of the cross-entropy method.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.