Energy-Based Policy Orchestration

Weirui Ye, Yen-Jen Wang, Elvis Hsieh, Zhiyang Dou, Shibo Zhao, Pieter Abbeel, Yilun Du, Leslie Pack Kaelbling, Tomás Lozano-Pérez

November 2026

The real G1 walks to a table, picks up a box and carries it.

TL;DR EPO learns, for each frozen humanoid whole-body policy, an energy that predicts whether a commanded goal stays feasible and gets reached, then refines goals and runs the lowest-energy policy at runtime. On a simulated Unitree G1 it cuts standing-manipulation tracking error from 0.461 to 0.420 against a learned mixture-of-experts scheduler while switching policies about 7x less often, and it runs on the real robot without fine-tuning.

What they do

Hierarchical controllers usually map a planner’s target straight to the goal of one low-level policy. On a humanoid, slips, pushes or loads can make that goal infeasible faster than the planner updates. EPO treats the interface as a runtime choice of which frozen policy to run and which goal to give it, scored by a learned energy per policy.

How it works

Policy family. Four goal-conditioned whole-body policies for a 29-DoF Unitree G1, trained with PPO in Isaac Lab: a general and a robustness-oriented loco-manipulation policy, plus nominal and robust standing specialists. All share a 16D command over base velocity, torso height and end-effector pose.
Energy. Per policy, a feasibility model predicts the chance of no fall over 1 s, and a Gaussian density predicts the realized command, giving the chance of landing within tolerance. Energy is the negative log of their product.
Runtime. Three gradient steps on velocity and torso-height goals at 10 Hz, then selection at 50 Hz with a feasibility threshold and an accumulated-advantage threshold that suppresses switching.
Extension. A new policy needs only its own energy models (about 30 GPU-minutes); a mixture-of-experts gate must be retrained.

Results

Loco-manipulation: tracking error 0.542 vs 0.568 for the robust policy, at 96.8% vs 97.7% weighted survival.
Standing: EPO-M4 error 0.420 vs 0.461 for MoE-M4 (survival 97.7% vs 98.3%), with 2.08 vs 15.46 switches per episode.
Removing the feasibility term drops standing push survival from 95.3% to 58.6%.
Hardware: box pick-up, basketball strikes and door opening without fine-tuning; in a preliminary gentle-push test EPO-M2 recovered 5/5.

Limitations

The energies may be miscalibrated under distribution shift or for newly added policies. Runtime grows with the number of policies (24.3 ms for four). EPO gives up about 2 points of push survival while walking, does not replace action-level safety filters, and the hardware results are teleoperated demonstrations rather than a quantitative evaluation.

Videos

The real G1 walks to a door and pulls it open.

Robust tracking on hardware, with basketballs thrown at the robot.

Figures

From Figure 1: EPO replaces the fixed planner-to-policy map with runtime optimization over a frozen policy family. It... — From Figure 1: EPO replaces the fixed planner-to-policy map with runtime optimization over a frozen policy family. It refines each policy's goal by energy minimization and executes the lowest-energy policy-goal pair.

From Figure 2: Precision-robustness trade-offs in simulation, averaged over flat, rough and stair terrain. The x-axis... — From Figure 2: Precision-robustness trade-offs in simulation, averaged over flat, rough and stair terrain. The x-axis is normalized tracking error, the y-axis weighted survival (mean of Normal, Extreme and Push survival). Left: loco-manipulation. Right: standing manipulation.

From Figure 5: Humanoid loco-manipulation tasks: stairs, rough and slippery terrain in simulation; robust tracking, b... — From Figure 5: Humanoid loco-manipulation tasks: stairs, rough and slippery terrain in simulation; robust tracking, box pick-up and door opening on hardware.

Abstract

Hierarchical control scales robot behavior by separating task-level planning from low-level control. In dynamic, high-dimensional tasks, however, a fixed planner-to-policy interface can fail when contacts, balance, or disturbances evolve faster than high-level policy updates. Execution therefore requires choosing not only which low-level policy to run, but also which policy-specific goal should be given to it. We formulate this problem as policy orchestration: runtime selection of a policy-goal pair that is likely to remain feasible while tracking the planner target. We develop EPO, an energy-based policy orchestrator for modular goal-conditioned policy families. For each policy, EPO learns a composite energy over low-level goals, where lower energy corresponds to higher feasibility and more accurate target realization. At runtime, EPO refines each policy's goal and executes the lowest-energy policy-goal pair. Because energies are policy-specific, new policies can be incorporated by learning their own energy models, without retraining the existing policies or a centralized scheduler. In simulation, EPO improves the empirical precision-robustness trade-off by refining policy-specific goals before selectively switching among policies based on energy scores. We further demonstrate sim-to-real deployment of the same frozen policies and energy models on real humanoid loco-manipulation tasks. Project website: https://yewr.github.io/epo

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong