D&A: Decoupling Description and Assessment Enables VLMs as Zero-Shot Robotic Reward Models

Marius Memmel, Shawn Yin, Urvi Rutia, Angad Wadhwa, Eldon Li, Gavin Angeloff, Ankit Goyal, Jesse Zhang, Dieter Fox, Fabio Ramos, Anqi Li, Abhishek Gupta

November 2026

Bimanual YAM policy trained with D&A rewards moving the red book from the shelf to the gray box.

TL;DR A prompt-only recipe that turns an off-the-shelf VLM into a robot reward model by first describing the video without seeing the task, then scoring it against an explicit success criterion. Its rewards lift real-robot online RL on a DROID Franka from 10% to 100% success.

What they do

Asked directly for per-frame task progress, VLMs tend to assume the robot succeeded and hallucinate progress on failed runs. D&A (Describe & Assess) is a zero-shot, purely prompt-based reward model that splits the job in two so the model cannot read the task into the video. No robot data or finetuning is needed, so it works on any promptable VLM.

How it works

Describe. An entity extractor lists every object in the scene, task-relevant or not. A describer then narrates the whole video, downsampled to about 8 key frames, using those entities but without seeing the instruction.
Assess. A separate call gets the instruction and the description, first writes what the world looks like when the task is done, then scores each timestep against that criterion. Scores in [0,1] are interpolated back to every frame.
Feedback. The assessor can also output a failure analysis with corrections, used as feedback between attempts of a code-as-policies planner.
Rewards drive online RL (DSRL on π0) or offline RL by advantage conditioning or filtering on π0.5. The default base model is Gemini 3 Flash.

Results

Online RL, DROID Franka, “put the fruit in the bowl”: success goes from 10% to 100% in about 90 minutes; with TOPReward rewards it is 5%.
Offline RL on a bimanual YAM moving a book to a box: 100% vs 20% for BC, 80% for TOPReward and 85% for Robometer. On a book-extraction task, 95% vs 80% for BC, finishing in 18.8 ± 1.8 s vs 28.5 ± 9.2 s.
On their 54-trajectory, 14-task D&A-Bench, D&A ranks trajectories best (Kendall τ 0.67 vs 0.56 for Robometer, 0.24 for TOPReward).
Rich feedback lifts code-as-policies on LIBERO-Goal from 40.6% to 51.4%.

Limitations

It inherits the latency, token cost, context limits and occasional hallucinations of closed frontier VLMs. Rewards are only as fine as language can describe, which is hard for precise spatial relations or distances.

Videos

D&A-filtered policy nudging out the book "Thinking, Fast and Slow".

The same book task with data filtered by TOPReward rewards instead.

Figures

From Figure 1: Confirmation bias in naive VLM reward estimation. Zero-shot querying hallucinates progress on failed t... — From Figure 1: Confirmation bias in naive VLM reward estimation. Zero-shot querying hallucinates progress on failed trajectories, so success and failure reward curves overlap; D&A separates them, shown on "pass me some M&Ms".

From Figure 3: Overview of D&A. A describer narrates the robot video without seeing the instruction, grounded in a li... — From Figure 3: Overview of D&A. A describer narrates the robot video without seeing the instruction, grounded in a list of scene entities. An assessor then writes a success criterion for the instruction and scores each timestep of the description against it, giving dense per-frame rewards between 0 and 1 and language feedback.

From Figure 4: Real-world RL results. Left: online RL on "put the fruit in the bowl", where TOPReward gives high reward to picking up both the ice cream cone and the banana. Middle, right: offline RL on two bimanual book tasks, where behavior cloning on suboptimal data hurts and D&A rewards give a higher success rate and faster completion.

Abstract

Vision-Language Models (VLMs) hold promise for zero-shot video task-progress estimation in robot learning, yet existing frameworks frequently suffer from confirmation bias, hallucinating progress in failed executions due to implicit success assumptions or relative temporal ranking designs. Naive prompting of frontier models collapses into a binary success detector that fails to capture nuanced, non-linear progress and failures. While post-training approaches can mitigate this through explicit training, they demand expensive real-world robotics data and require computationally intensive post-training for every individual base model. To resolve these limitations, we introduce D&A (Describe & Assess): our approach decouples progress estimation into an objective, task-grounded textual description of scene dynamics followed by an assessment phase using explicit rubrics. By compelling the VLM to synthesize an explicit, text-based definition of absolute success before scoring, the framework anchors evaluations to an external, global criterion rather than the video's internal timeline. Empirical results demonstrate that our method calibrates zero-shot estimates and mitigates confirmation bias, providing fine-grained, partial progress metrics that accurately penalize failed executions while correctly interpolating successful runs. Being purely prompt-based, D&A generalizes across tasks, embodiments, and benchmarks---matching or surpassing post-trained reward models---and its reward estimates enable efficient downstream reinforcement learning.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong