Training Reasoning Models in Simulation to Natively Use Real Robots

TL;DR An open-weight 27B reasoning model learns to drive robot arms the way computer-use agents drive a PC (look at camera images, reason, output the next end-effector pose), trained with RL only in simulation and without demonstrations. On a real dual-arm robot it never saw in training it succeeds in 18 of 30 trials; the base model succeeds in none.
What they do
Robot-use agents let a vision-language model operate a robot directly, but so far only proprietary frontier models do this well. The authors train an open-weight reasoning model, Qwen3.8-27B, to use robots with reinforcement learning alone, entirely in simulation and without demonstrations, then deploy it zero-shot on a real dual-arm YAM.
How it works
- Interface. Each round the model sees camera frames and the arm state as JSON, reasons in a chain of thought, and outputs the change in each end-effector pose. Inverse kinematics and the robot’s own controller execute it. There are no perception tools, planners or skill libraries; only the prompt’s robot description changes between robots.
- RL. A binary task-success reward with GRPO-style group-normalized advantages, no PPO clipping, and fully asynchronous rollouts corrected by score centering. The PEAR infrastructure builds on veRL and LMCache; training took about 50,000 H100-equivalent GPU-hours.
- Curriculum. The base model succeeds in about 2% of LIBERO episodes, so training starts on three LIBERO suites, adds LIBERO-Long, then two dual-arm RoboDojo tasks.
Results
- LIBERO success rises from 1.6% to 74.4%, against 76.2% and 80.6% for GPT-6.1 Sol and GPT-6 Astra at maximum effort.
- Given a different task in a training scene (LIBERO-Pro), pi0.5 drops to 0.75% while the model keeps 57.4%; pi0.5 still leads on reworded instructions and restyled objects.
- Zero-shot on RoboLab-120, 37.3% versus 28.0% for the pi0.5 leaderboard entry.
- Real YAM: 18/30 trials for the final model, 7/30 for the LIBERO-only checkpoint, 0/30 for the base model. No measurable loss on nine reasoning, coding and vision benchmarks.
Limitations
It is slow: a round of reasoning takes about 4.3 s on a GB300 and occasionally over half a minute. Training covers two simulators and few tasks, harder tasks need the embodiment described in detail in the prompt, and the model outputs one pose per round rather than action chunks. On the full RoboDojo benchmark, 68.5% of episodes fail by exhausting the 256k-token context.
Figures




Abstract
Robot-use agents (RUAs) have emerged as a promising approach to general-purpose robot control. However, strong performance has so far been achieved only by frontier proprietary models such as GPT-6 Astra and Claude Fable 5.2, whereas open-weight models remain substantially less capable. We show that an open-weight 27B reasoning model can be trained to use real robots natively through reinforcement learning, entirely in simulation and without demonstrations. Like a computer-use agent (CUA), the model observes the environment, reasons in a chain of thought, and generates the next end-effector pose, without any harness. We build a Post-training Engine for Agentic Robotics (PEAR) and use it to train the model on simulated robot tasks at scale with robot-use reinforcement learning (RURL), mirroring how frontier models acquire computer use. We find that the skills acquired in simulation generalize well beyond the training tasks: they transfer to an unseen robot, to the real world, and to novel instructions. After training on LIBERO and RoboDojo in simulation, the model succeeds in 60% of our trials on a real dual-arm robot, whereas the base model succeeds in none. On LIBERO, training raises the success rate from 1.6% to 74.4%, approaching GPT-6 Astra and GPT-6.1 Sol at maximum reasoning effort. Compared with $π_{0.5}$, a vision-language-action model fine-tuned on demonstrations, our model is substantially more robust to changes in the task itself. On LIBERO-Pro, $π_{0.5}$ succeeds in only 0.75% of episodes when assigned a new task in a familiar scene whereas our model succeeds in 57.4%. Finally, the model retains its general capabilities, with no measurable degradation on nine reasoning, coding and vision benchmarks. These results highlight robot-use reinforcement learning as a promising component of foundation-model post-training.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
