APG-Planner: Learning Vision-Based High-Speed Trajectory Planning via Analytic Policy Gradient

TL;DR Train a depth-image trajectory planner for fast quadrotor flight by backpropagating episode losses through its own planned trajectories, using differential flatness instead of a dynamics simulator. It trains in about 30 minutes on one GPU, keeps 92-94% success at 7.5-8.2 m/s mean speed in simulated forests where PPO gets 60-64% at about 5 m/s, and flies real forests at up to about 11 m/s.
What they do
Learned planners for vision-based high-speed flight are usually trained by imitating an expert, which is limited by the demonstrations, or by guidance from a planning cost that stays local to each decision. Model-free RL optimizes whole episodes but estimates its gradients from noisy samples. APG-Planner makes an entire receding-horizon episode differentiable and trains the planner with analytic policy gradients. Differential flatness supplies the state transitions, so no low-level dynamics model or calibrated simulator is needed.
How it works
- Planner. A CNN encodes a 40x64 depth image. An MLP adds velocity, acceleration, roll, pitch and a goal vector and outputs a terminal position, velocity and acceleration, joined to the current state by a 1 s quintic polynomial.
- Flatness-based rollout. The state 0.1 s along each plan is reconstructed analytically through the quadrotor’s flatness map, and a new depth image is rendered there. Fifty such steps form one differentiable episode.
- Loss and training. Goal progress, jerk and obstacle clearance along each planned segment. Gradients flow across steps through position, velocity and acceleration. Training takes about 30 minutes on one RTX 4080 SUPER.
- Deployment. A geometric controller tracks the plan; replanning runs at 10 Hz, about 13 ms per plan.
Results
- In Flightmare forests (50 maps per density, 40 m courses), it reaches competitive or higher success than EGO, AGILE and YOPO, with lower snap, especially at high speed.
- Full APG reaches 94%/92% success (sparse/dense) at 8.16/7.47 m/s mean speed. Cutting gradients between replanning steps keeps success (94%/88%) but slows flight to about 2 m/s; PPO on the same objective gets 60%/64% at about 5 m/s.
- On a 25 cm quadrotor with onboard visual-inertial odometry and stereo depth, it flew collision-free through two forests at up to about 11 m/s. With a unicycle flatness model, the same framework drove a Clearpath Jackal through a forest.
Limitations
Training assumes the controller tracks the plan exactly. Real-world runs are shown as demonstrations, with a peak speed but no success rates or baseline comparisons. The method needs a differentially flat system.
Figures




Abstract
Vision-based high-speed navigation requires a trajectory planner that can reactively generate smooth, collision-free trajectories from onboard visual observations. Deep reinforcement learning offers a promising approach to vision-based trajectory planning, as it enables the optimization of sequential decision-making policies in complex environments. However, model-free policy gradient methods rely on sample-based policy gradient estimation, which yields noisy gradient estimates and inefficient training. To this end, we propose a framework for learning trajectory planning policies using the analytic policy gradient (APG) method and use it to train APG-Planner, a vision-based high-speed trajectory planner. The proposed framework leverages differential flatness to analytically reconstruct vehicle states from planned trajectories, thereby enabling differentiable trajectory-level state transitions without requiring an explicit low-level dynamics model. By iteratively performing trajectory planning and differentiable state transitions, we construct a differentiable episode rollout and analytically compute policy gradients from episode-level losses, improving both training efficiency and policy performance. After approximately 30 minutes of training on a single GPU, APG-Planner demonstrates comparable or superior performance to existing baseline methods in simulation and achieves speeds of up to 11 m/s in real-world flight experiments. We further validate the proposed framework on a skid-steer ground robot, demonstrating its applicability to differentially flat robotic systems beyond quadrotors.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
