PGX: Physically Grounded Execution for Long-Horizon Humanoid Loco-Manipulation
TL;DR PGX wraps a fine-tuned VLA in a predicate-gated state machine and world-frame locking, so on a humanoid the VLA only runs the manipulation stages of a long task. On simulated coffee fetching it lifts GR00T N1.6 from 35% to 80% success with unchanged weights, and it also runs on a real Unitree G1.
What they do
A fine-tuned VLA can run single manipulation skills on a humanoid but fails on long tasks that mix walking, grasping and device use: it loses track of the stage, its own base commands make the body drift, and walking dominates the demonstrations. PGX leaves how to manipulate to the VLA and decides when and where to act from physical state, without retraining.
How it works
- Predicate-gated state machine. Each stage has an instruction, an executor (navigation controller, VLA or a wait), a completion guard, a time budget and a retry policy. Five predicates (grasped, placed, triggered, at-stance, dropped) gate transitions; a failed guard triggers a retry, and stage confusion triggers a reverse transition to an earlier stage.
- One rule, three uses. The same predicates cut demonstrations into manipulation segments for stage-trained models, gate execution online and score stage progress.
- World-coordinate locking (WCL). Navigation goals are set relative to the object’s current pose. During manipulation the executor overrides the VLA’s base-velocity commands and holds a world-frame stance, and precise targets such as a button are stored in the world frame.
- Backbones are π0.5 and GR00T N1.6 on a Unitree G1 with Dex-3 hands, tested in Isaac Lab Arena and on hardware.
Results
- Simulated coffee fetching (nominal/random start): π0.5 goes from 0%/0% to 60%/55% success, and GR00T from 35%/20% to 80%/80%. Pressing the button with the world-anchored primitive raises press rates to 95-100%.
- Averaged over stage progress and success on box transport, mug placement and coffee fetching: 35% to 78% (GR00T) and 17% to 82% (π0.5).
- Most of the gain comes from the executor itself. Stage-wise training changes π0.5 success by at most 5 points.
- Under injected base pushes, station keeping cuts the p90 per-stage drift from 54 cm to 9 cm.
- Real G1 (44 demonstrations): grasping transfers best, and placing and re-grasping fall furthest below simulation.
Limitations
Stage transitions make execution slower. Each task’s stage graph is given, not learned, and the real-robot predicates read Vicon motion capture. The real experiment is one task and reports per-stage completion, not end-to-end success.
Videos
Figures




Abstract
Robotic vision-language-action (VLA) models can execute diverse individual manipulation skills, yet they still struggle with long-horizon loco-manipulation tasks, where temporal structure, skills, and precision are tightly coupled within a monolithic model. We propose Physically Grounded Execution (PGX), a framework that offloads temporal and precision control to two classical training-free components, allowing the VLA to focus on stage-specific skill dynamics. PGX decomposes long-horizon loco-manipulation into three layers: (i) a predicate-based state machine governs task progression, using physical predicates to gate phase transitions, trigger recovery, and coordinate navigation and waiting. Shared predicate semantics connect offline demonstration segmentation, online execution, and stage-wise evaluation; (ii) VLA skills operate within active manipulation phases, with flat or stage-trained models concentrating on task-relevant actions; and (iii) world-coordinate locking (WCL) maintains spatial consistency through base station keeping, object-relative positioning, and world-anchored contact targets. We demonstrate the effectiveness of PGX with extensive experiments conducted on three simulated tasks and one real-world task using a Unitree G1 humanoid robot for long-horizon loco-manipulation, showing improvements in stage progression and task success rates. Project page is available at: https://kl-research.github.io/pgx
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
