Recover, Don't Just Continue: Learning from Feedback for Physical Failure Understanding and Correction
TL;DR A multimodal "corrector" model, trained on failures injected into successful simulated trajectories, checks each finished subtask, names the observable cause of a failure and writes a recovery instruction for a frozen executor. On a real Piper arm it raises macro task success from 45% to 62% for π0.5 and from 51% to 70% for LingBot-VLA.
What they do
When a robot fails partway through a task, a planner that only sees the instruction tends to repeat the same command. Embodied-Corrector is a multimodal model that runs after each subtask. It reads the before and after observations and the execution trace, then outputs four fields: Judge (did the subtask complete?), Cause (the failure, described from what is observable), Sub (advance, or redo which subtask) and Fine (step-by-step recovery). A fixed GPT-5.5 planner or a frozen VLA then carries out the correction from the current state, without resetting the scene.
How it works
- Controlled failure data. They start from successful RoboCasa365 and OmniBehavior trajectories, clone the simulator at a subtask boundary, change exactly one factor and replay the subtask. The factors are navigation goal, reachability, grasp pose, gripper timing, container articulation, a skipped prerequisite and the wrong target instance. They keep only failures that are visible in the observations and recoverable, and a corrective replay supplies the Fine target. The result is 1,583,014 rounds from 148,415 rollouts.
- Evaluation-first SFT. Starting from RynnBrain-8B, the model first learns Judge and Cause, then learns Sub and Fine conditioned on correct Judge and Cause.
- On-policy distillation (NEPO). At deployment the model plans from its own Judge and Cause, which may be wrong. Training therefore compares planning-token probabilities under its own prefix and under the correct one, reinforcing or suppressing tokens. Negative signals are damped where the model’s own distribution has high entropy.
Results
- EmbodiedDiag-Bench: Judge accuracy is 79.6% (navigation) and 81.2% (manipulation), against 52.8% and 57.7% for GPT-5.5. Cause accuracy is 58.2% and 54.6%, against 39.8% and 43.2%.
- Frozen π0.5 in RoboCasa365: composite-task success rises from 7.1% to 12.5% on seen tasks and from 1.2% to 7.3% on unseen ones. Short atomic tasks get worse.
- Feeding the corrections back to a GPT-5.5 code planner improves all ten OmniBehavior and LIBERO-PRO conditions over the untrained brain, by 13.9 points on average.
- Real Piper arm, five tabletop tasks, 400 episodes: every task improves. Spatial rearrangement stays hardest, at 45% and 55% with correction.
Limitations
Real-robot tests cover five tabletop tasks on one arm, with up to 10 correction attempts per episode. Rewording instructions that a frozen VLA already handles can introduce errors, a gap in language alignment between the corrector and the VLA. Observation noise and state uncertainty still make it hard to judge outcomes. The training data come entirely from simulation.
Videos
Figures




Abstract
Multimodal large language models are regarded as the cognitive foundation of embodied intelligence, but planning from instructions alone cannot make them reliable. During execution, actions deviate from intended effects, constrained by the embodiment and environment. An embodied brain must understand why execution deviates, identify what can be reliably realized, and revise decisions using interaction experience. These correspond to causal failure localization, capability-constrained adaptation, and experience-driven revision. Current embodied foundation models largely lack these capabilities. To address this, we introduce Embodied-Corrector. We construct failure-recovery data by injecting perturbations at subtask boundaries of successful trajectories, replaying affected subtasks, and keeping failures that are observable from public evidence and recoverable from the current state. We align actions, consequences, failure causes, and recoveries into Judge-Cause-Sub-Fine supervision. Training uses evaluation-first SFT: the model learns Judge and Cause first, then Sub and Fine conditioned on accurate prefixes. Since deployment uses self-generated prefixes, evaluation errors propagate into recovery planning, and response-level rewards are too coarse to locate affected planning tokens. We therefore use OPD to internalize the planning distribution under accurate prefixes into that under self-generated ones: the model generates plans from self-generated prefixes, while a contrastive forward pass with accurate prefixes compares token probabilities to reinforce or suppress planning tokens. At deployment, this branch is removed. Experiments show stronger failure analysis and corrective planning: with a frozen VLA, composite task success improves by 6.1 percentage points; with fixed GPT-5.5 execution, manipulation and navigation tasks improve by 13.9 percentage points.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
