DreamVLA: From Generated Human Motions to Humanoid Vision-Language-Action Policies
TL;DR Generated human motions guide an RL expert in simulation, whose successful rollouts and recoveries from the student's failures train a GR00T policy for a Unitree G1, with no task-specific teleoperation. Three DAgger rounds lift simulated picking from 0% to 54%, while the same amount of plain expert data stays at 0%.
What they do
Teleoperating a humanoid to collect whole-body demonstrations for a vision-language-action (VLA) policy is slow. DreamVLA replaces the operator with generated human motion: a motion model proposes how to do the task, an RL expert in simulation turns that into physically executed interaction, and the result is distilled into GR00T. The study is entirely in simulation, on a Unitree G1 and four tasks: pick from a platform, open a drawer, press a button, and walk and pick.
How it works
- References. Following DreamControl, OmniControl samples human motions conditioned on text and on joint targets in space and time. They are retargeted to the G1 with OmniRetarget and augmented with HumanoidMimicGen.
- Residual expert. A privileged RL policy co-tracks the humanoid reference and the object, adding residual corrections in the motion-token interface of the SONIC whole-body controller. Hands are binary open/close.
- Offline dataset. Successful expert rollouts are saved in LeRobot format with egocentric images, proprioception and language, and used to fine-tune GR00T.
- Reference-conditioned DAgger. When the student fails, a random state from its rollout is restored and the expert resumes with the rest of the same reference. Only recoveries that succeed are added.
Results
- With 200 initial trajectories and 200 corrective ones per round, three DAgger rounds raise student success from 0% to 54% on pick, 51% to 60% on drawer, 59% to 69% on button and 0% to 18% on walk and pick (100 rollouts each).
- A control with 800 expert trajectories and no recoveries reaches 0%, 54%, 57% and 0%.
- Expert: residual token RL reaches 88.6%, 74.0%, 81.0% and 75.95% on the four tasks. Nominal SONIC tracking gets 0%, 30.0%, 68.0% and 17.30%; absolute token RL gets 0% on all four.
Limitations
Simulation only, four tasks, one embodiment, and no visual domain randomization; hardware deployment is left to future work. The student stays well below the expert, most of all on walk and pick. Generated motions can be incompatible with the robot’s kinematics or contacts, and task-specific guidance, filtering and reward weights are still needed.
Figures
Abstract
Training vision-language-action (VLA) policies typically depends on substantial robot demonstration data, making human teleoperation a costly and cumbersome bottleneck, particularly for whole-body humanoid interaction. Reinforcement learning in simulation offers an alternative for scaling this data, but designing task-specific rewards remains challenging. To address this, we propose DreamVLA, which uses generated motion trajectories to guide reinforcement learning (RL) in simulation, converting them into reliable task execution trajectories. These trajectories train VLA models in simulation without new task-specific teleoperation data. Our central contribution is a reference-conditioned DAgger distillation procedure: after a failed learner rollout, we restart the RL expert from a sampled learner state with the same reference suffix and aggregate successful corrective trajectories, addressing the distribution shift left by offline imitation. We evaluate picking from a platform, opening a drawer, pressing a button, and walking and picking from a platform, with ablations of DAgger iterations, residual RL, and generated dataset scale. Project website: ember-center-berkeley.github.io/DreamVLA.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.