SUGARv2: Learning Diverse Visual Humanoid Loco-Manipulation Skills from Synthesized Interaction References

Tianshu Wu, Tianxiang Gui, Yue Chen, Tingxu Liu, Ziye Peng, Guangjie Yang, Xiangqi Kong, Ye Shi, Jingya Wang, Hao Dong

November 2026

Real-world Tabletop Pickup on the humanoid, seen from front, side and egocentric cameras.

TL;DR Synthesize humanoid loco-manipulation references from reusable task templates instead of teleoperation, then train camera-based policies with one shared imitation reward. Across five pickup and carrying tasks SUGARv2 reaches 91.0% success in simulation and transfers zero-shot to a real Unitree G1.

What they do

Teaching a humanoid to pick up and carry objects from its own camera usually takes teleoperated demonstrations or a hand-tuned reward for each task. SUGARv2 replaces both with synthesized interaction references. A language model (GPT-6 Astra) writes one reusable template per task. The template is adapted to each robot, object and goal layout to produce whole-body robot motion, an object trajectory and hand contact labels. One shared imitation reward then trains a visual policy per task in simulation, which is deployed on a Unitree G1.

How it works

Templates. A template is an ordered list of keyframes, each giving a whole-body pose, robot-object and hand-object relations, and which hands touch the object. For a sampled configuration, the keyframes are adapted under reachability and scene constraints. An off-the-shelf motion generator fills in continuous motion, and the object trajectory follows the hands.
Privileged teacher. It outputs latent commands for a frozen pretrained whole-body controller plus open/close hand commands. It is trained on a mimic reward for tracking, interaction and regularization, which also keeps the object in camera view.
Visual student. It takes egocentric RGB and proprioception. It is first trained on teacher actions at the states it visits, then with PPO on the same mimic reward while the distillation loss stays on. Objects, backgrounds, lighting and camera pose are randomized.

Results

Across five tasks the mean success in simulation is 91.0%. VIRAL gets 39.6% and fine-tuned GR00T N1.7 gets 23.4%, both trained on the same references and teachers.
Each reference takes 0.4 min to synthesize, versus 36.1 min with GRAIL, and teacher success is 98.0% versus 62.4%. Policies trained on GRAIL references average 28.2%.
With unseen backgrounds and objects combined, mean success is 83.2%. Without distillation success drops to 0%, and without the RL stage it is 41.4%.
Zero-shot on the real G1: tabletop pickup 9/10, floor-to-table 6/10, box carrying 8/10, basket transport 8/10, same-table 7/10.

Limitations

The tasks are short-horizon in simple scenes, with one policy per task. After a missed grasp the policy often carries on without retrying.

Videos

Method overview: template instantiation into references, then privileged teacher training in simulation.

Tabletop Pickup in simulation, from synthesized references to skill execution in randomized scenes.

Figures

From Figure 2: Overview of SUGARv2. Left: a task's interaction template is instantiated for different spatial configu... — From Figure 2: Overview of SUGARv2. Left: a task's interaction template is instantiated for different spatial configurations, giving keyframes and timing that guide reference synthesis (Floor-to-Table shown). Right: a privileged teacher learns to execute the references through a frozen pretrained whole-body controller with a reference-based mimic reward; the visual policy uses the same controller and learns by teacher distillation and reinforcement learning with the same reward.

From Figure 3: Synthesized interaction references for the five tasks. Overlaid poses show references for different sp... — From Figure 3: Synthesized interaction references for the five tasks. Overlaid poses show references for different spatial configurations, with later poses drawn more opaquely.

From Figure 6: Real-world execution: (a) Tabletop Pickup, (b) Floor-to-Table, (c) Box Carrying, (d) Basket Transport,... — From Figure 6: Real-world execution: (a) Tabletop Pickup, (b) Floor-to-Table, (c) Box Carrying, (d) Basket Transport, (e) Same-Table.

Abstract

Scaling up visual humanoid loco-manipulation skill acquisition requires reducing the human effort needed as tasks and spatial configurations expand, particularly in demonstration collection and task-specific training design. We present SUGARv2, a unified framework for learning diverse visual humanoid loco-manipulation skills through interaction reference synthesis and a shared training pipeline. To obtain diverse references at low cost, we adapt reusable task templates to different spatial configurations, synthesizing coordinated robot and object motions. To reduce task-specific reward design and training effort, we use reference-based imitation as a common learning objective across tasks. We first train privileged teachers to execute these references, then train visual policies through teacher distillation and reinforcement learning with the same reference-based mimic rewards, using diverse object instances and visual domain randomization to support visual generalization and sim-to-real transfer. Across five humanoid loco-manipulation tasks, SUGARv2 achieves a mean success rate of 91.0% in simulation. Deployment on a real humanoid robot further demonstrates successful sim-to-real transfer for whole-body manipulation. These results demonstrate the framework's potential for learning diverse visual humanoid loco-manipulation skills.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong