← Paper Digest

Heavy Metal Acrobatics: Learning Dynamic Control for Skid-Steer Loaders

Pierre-Jean Lorentz, Pascal Egli, Edo Jelavic, Cliff Li, Ramitha Sundar, Harikrishnan Suresh, Paul Montgomery, Lee Redden, Matthias Müller, Ozan Sener, Vladlen Koltun, Marco Hutter

November 2026

TL;DR RL policies trained in simulation, with a neural model of the hydraulic drive fitted to real driving data, make a 5-tonne skid-steer loader swing up onto its front wheels, balance, drive, spin and land softly. Trained only on flat ground, it drives over 75 cm bumps that stop an MPC baseline.

What they do

A skid-steer loader has a hydraulic drivetrain with strong delays and nonlinearities, and it steers by giving its left and right wheels different speeds. The authors train RL policies in simulation that make a 5-tonne John Deere 332G rock itself up onto its two front wheels, balance there while driving, spinning or moving the arm, and land back on four wheels. Until now, only a few elite human operators had done these maneuvers. The policies run zero-shot on the real machine from a laptop CPU.

How it works

  • Retrofit. A computer sends the joystick commands over the CAN bus. Sensing is the factory wheel encoders plus two added IMUs. A wheelie bar and wheels on the bucket protect the machine if it tips.
  • Simulation from simple measurements. The rigid-body model is a cuboid with four cylinder wheels, measured with a tape measure and truck scales. A small MLP actuator model maps 1.8 s of command and wheel-speed history to the next wheel speed. It is trained on 1.5 h of random commands plus 20 min of balancing data.
  • Separate skills. Swing-up, stabilize-and-balance, velocity tracking and soft landing are each trained with PPO, using randomized arm angle, center of mass and inertia, sensor noise and pushes. Simple rules switch between them.

Results

  • Swing-up succeeded in all 1000 randomized simulation runs. On the machine it was reliable at trained arm angles; with the arm higher, the bucket touched the ground in 3 of 14 trials.
  • Balancing in place oscillated by ±1.6°, on par with MPC. RL tracked up to 4 rad/s, while the MPC fell beyond 3.5 rad/s and cannot do the swing-up or landing at all.
  • Trained only on flat ground, the policy crossed 75 cm bumps forward and backward. The MPC stayed balanced but could not get over them.

Limitations

The skills are composed with hand-written switching rules, and the policies see no terrain (no camera or LiDAR). Some sim-to-real gap remains: balancing pitch differs by about 2°. At the fastest spin the RL policy oscillated more than the MPC. Apart from the swing-up, hardware results are shown as time series rather than success rates over many trials.

Figures

From Figure 1: Acrobatic maneuvers trained with RL in simulation and run on the 5-tonne skid-steer loader. (A) Swing-...
From Figure 1: Acrobatic maneuvers trained with RL in simulation and run on the 5-tonne skid-steer loader. (A) Swing-up from four onto two wheels. (B) Driving over rough terrain on two wheels. (C) Balancing while moving the arm. (D) Spinning in place on the front wheels. (E) Soft landing.
From Figure 5: Method overview. (A) Policies are trained in a simulation built from a learned actuator model and a ra...
From Figure 5: Method overview. (A) Policies are trained in a simulation built from a learned actuator model and a randomized rigid-body model, then deployed on the real machine. (B) Wheel speed for the same joystick input is nearly the same with the wheels in the air and on the ground. (C) Simulated vs. real wheel speed for ramps, sinusoids and balancing commands. (D, E) Hand-measured collision geometry and the inertia approximation.
From Figure 3: Hardware deployment. (A, B) Swing-up in simulation and reality; the stabilize-and-balance controller t...
From Figure 3: Hardware deployment. (A, B) Swing-up in simulation and reality; the stabilize-and-balance controller takes over when pitch reaches -30 degrees. (C, D) Soft landing compared with a zero command, after which the front wheels rebound and the machine rocks. (E) Balancing in place with RL and the MPC baseline. (F) Velocity tracking with RL and MPC, raised step by step up to the fastest target each controller tracks without falling.
From Figure 4: Robustness analysis. (A) Balancing while the arm moves up and down, RL vs. MPC. (B) Spinning at differ...
From Figure 4: Robustness analysis. (A) Balancing while the arm moves up and down, RL vs. MPC. (B) Spinning at different rates, added as a rotation command on top of the policy actions. (C) Driving backward over 75 cm bumps; the MPC (green) cannot get over the first bump, even with a higher target velocity.
Abstract

Automation of heavy machinery offers the opportunity to improve productivity in mechanized industries such as construction, mining, and material handling. Yet, automation of such machines is challenged by their complex dynamics, in part because of hydraulic actuation that involves fluids, flexible elements, and nonlinear delays. A concomitant challenge is the size and weight of these machines, which can make experiments on the physical systems exceedingly dangerous. Here, we show that reinforcement learning (RL) in simulation, followed by transfer of the learned policies from simulation to reality, can successfully automate acrobatic maneuvers that push a heavy construction machine to the limits of its actuation and have only previously been accomplished by a small number of elite human operators in official world-record performances. We trained policies that perform autonomous acrobatic maneuvers on a John Deere 332G, an operational 5-tonne skid-steer loader. The policies can autonomously swing the machine up onto two wheels and, while balancing on two wheels, drive over rough terrain, trace out specified trajectories, spin around, and swing the bucket back and forth before landing it safely back onto four wheels. This work demonstrates that RL and sim-to-real transfer enable dynamic autonomous control of heavy machinery, directly contributing to greater automation in heavy equipment sectors.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.