← Paper Digest

Grana: Compositional Reward Modeling for Long-Horizon Robot Manipulation

Zi-ang Cao, Kaiyuan Zheng, Yinzhen Xu, Haonan Duan, Jiasheng Gu, Fengyuan Hu, Nan Zhu, Kun Lei, Kevin Lin, Ruijie Zheng, Shenyuan Gao, Fangqi Zhu, Zhe Zhang, Radha Poovendran, Linxi Fan, Guanzhi Wang, Yuke Zhu

November 2026

A real bimanual robot assembles the airplane model, seen from four cameras.

TL;DR A vision-language reward model picks the unfinished subtask from a language list, predicts its progress and adds credit for the finished subtasks. Pretrained on 200 tasks, it cuts zero-shot progress error by 20.7% over RoboMeter, and progress-weighted cloning raises real airplane-assembly success from 50% to 75%.

What they do

Grana is a progress-based reward model for multi-minute assembly tasks, where a failure such as dropping a screw should reset the current subtask without erasing credit for the finished ones. A vision-language model selects the unfinished subtask from an ordered list of language descriptions and predicts that subtask’s local progress p. With k subtasks finished out of K, whole-task progress is (k + p)/K. The model is pretrained on 200 mostly short tasks and then adapted to long ones.

How it works

  • Shared predictor. Qwen3.5-4B is called twice: once to select the subtask, once to predict progress for it. Each call sees eight timesteps from four cameras. LoRA adapters, a temporal transformer and a binned progress head are trained; the base model stays frozen.
  • Structured supervision. Annotators mark subtask boundaries and intervals of suboptimal or regressed execution, not numeric scores. Nominal frames advance progress 20x faster than suboptimal ones. A regression resets local progress to zero, while completed subtasks keep their credit.
  • Order-aware targets. When the model queries the wrong subtask, earlier subtasks get a target of 1 and later ones 0.
  • Policy use. Progress increments become rewards for offline RL (IQL, then BPPO), or per-step weights (1, 0.5, 0) for behavior cloning of a GR00T-2 world-action model.

Results

  • On 50 held-out tasks, zero-shot RMSE is 0.180 vs 0.227 for RoboMeter. Going from 15 to 200 pretraining tasks cuts RMSE by 35.3%.
  • On five long tasks (up to 18 subtasks), each adapted with 15 labeled trajectories, Grana has the lowest RMSE on every task, 16.1-31.6% below the best baseline. Moving from the YAM robot to the Tianji robot: 0.452 vs 0.556.
  • Offline RL with zero-shot rewards reaches 65.0% on star insertion and 66.7% on tube transfer, vs 50.0% and 41.7% for the best other reward source.
  • In the second round of weighted cloning, success is 75% vs 50% on airplane and 67% vs 42% on car assembly. Using Grana to trigger and guide MPC raises tractor assembly from 2/10 to 9/10.

Limitations

Not discussed in a section of their own. Composing progress assumes completed subtasks stay completed, and a wrong subtask selection still distorts the estimate. Because the model can use later frames, the policy experiments score recorded trajectories offline. Long tasks need annotated adaptation trajectories, and each policy result rests on 12 trials.

Videos

On a real rollout, Grana's predicted progress drops when the wing comes off; the baseline's does not.
Airplane assembly before post-training (the wing slips off) and after progress-weighted cloning with Grana.

Figures

From Figure 1: Grana is pretrained on diverse short-horizon tasks. On a long task it selects the current subtask from...
From Figure 1: Grana is pretrained on diverse short-horizon tasks. On a long task it selects the current subtask from language descriptions and predicts task progress from multi-view observations, trained with dense targets built from nominal, suboptimal and regressed execution. It is used for offline RL, progress-weighted behavior cloning and cross-embodiment reward prediction.
From Figure 2: A shared vision-language model selects the unfinished subtask from language descriptions, estimates pr...
From Figure 2: A shared vision-language model selects the unfinished subtask from language descriptions, estimates progress for that subtask, and combines the estimate with credit for the completed subtasks to give whole-task progress.
From Figure 3: A screw-insertion failure (A-B) and recovery (C-D) after six of nine subtasks. Subtask-only targets mi...
From Figure 3: A screw-insertion failure (A-B) and recovery (C-D) after six of nine subtasks. Subtask-only targets miss the failure and action-quality-only targets drop to zero; combining both resets insertion progress while keeping the 6/9 from completed subtasks, then tracks the recovery.
From Figure 7: Success rates of progress-weighted policy learning on airplane and car assembly over two iterations, c...
From Figure 7: Success rates of progress-weighted policy learning on airplane and car assembly over two iterations, comparing unweighted cloning, SARM2 weights and Grana weights. Each rate uses 12 trials.
Abstract

Robot reward modeling faces two challenges: generalizing across tasks and accurately tracking progress in long-horizon manipulation. General-purpose reward models demonstrate broad task transfer, but transfer from short-task pretraining to long-horizon tasks remains underexplored. Long-horizon progress models often rely on task-specific subtask classifiers or fixed action vocabularies, limiting the reuse of learned progress knowledge when adapting to new tasks. To address these challenges, we introduce Grana, a vision-language robot reward model that learns to identify unfinished subtasks and estimate their continuous progress using open-vocabulary subtask descriptions. We train a language-conditioned model to estimate progress in short-horizon tasks and individual subtasks of long-horizon tasks. For long-horizon tasks, we combine the selected subtask's predicted progress with credit for preceding completed subtasks to estimate whole-task progress. General progress prediction requires training the model across diverse tasks with progress supervision that accounts for imperfect execution. Elapsed-time proxies can misrepresent progress when relative subtask durations vary, or failures reverse local progress. We therefore introduce an annotation protocol that converts sparse subtask boundaries and action-quality annotations into dense progress targets, without requiring annotators to assign numerical progress scores. Pretraining with this supervision on 200 diverse, mostly short-horizon manipulation tasks enables zero-shot reward prediction on unseen tasks and improves progress prediction after adaptation to multi-minute, long-horizon tasks. Experiments demonstrate (i) generalizable reward prediction, including adaptation to five long-horizon tasks, (ii) benefits from pretraining and composing subtask progress estimates, and (iii) improved downstream policy performance.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.