LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning

Xuheng Zhang, Yiming Zhang, Jialin Gao, Hao Chen, Jiaming Liu, Ji Li, Runzhong Huang, Yinxi Wang, Chenyang Gu, Zihan Yang, Yaowei Liu, Zhonghao Yan, Ziyu Guo, Renrui Zhang, Peng Jia, Pheng-Ann Heng, Shanghang Zhang

November 2026

TL;DR A world action model that adds a latent expert between video prediction and action generation, trained to focus on robot-object interaction, plus a progress value that decides how many predicted actions to execute. Pretrained on over 14k hours, LaST0.5 reaches 99.5% on LIBERO and 63.8% on four real bimanual tasks against 52.5% for π0.5.

What they do

World action models (WAMs) predict future video and robot actions together, and usually pass global scene features straight to the action head. LaST0.5 inserts an intermediate “latent expert” that turns the predicted future into a few tokens focused on robot-object interaction, and it uses a predicted task-progress value to decide how much of each action chunk to execute.

How it works

Three experts, one attention stream. A Mixture-of-Transformers joins a world expert (Cosmos-Predict2.5-2B, predicting 12 future frames), a latent expert (initialized from Janus) and a flow-matching action expert. A blockwise-causal mask enforces video, then latent, then action over 3 aligned segments.
Interaction-centric latent targets. The last future frame of each segment is overlaid with the projected end-effector path and a Grounding DINO box on the target object, then encoded by a frozen SigLIP. The latent expert matches these targets with a cosine loss; the annotations are needed only for training.
Adaptive world-to-action transfer (AWAT). A small value head on the world expert’s features predicts progress per segment. The robot executes actions only while progress rises, then re-infers; if progress starts below the previous estimate, several futures are sampled and the best is kept.
Pretraining. 2.48M episodes and 14,892 hours, 61% of it handheld-gripper (UMI) data.

Results

LIBERO: 99.5% average (99.2% without AWAT). RLBench, ten tasks: 88.0%, against 82.0% for LaST0 and 65.0% for π0.5.
Real bimanual Tianji Marvin, four tasks, 150 demos each, 20 rollouts: 63.8% average, against 52.5% for π0.5 and 43.8% for Cosmos Policy. Screw insertion succeeds 50% of the time and box sealing 30%.
Under shifted object positions, average success is 40.0% against 15.0% for π0.5.
Ablation on LIBERO-Spatial: removing the world expert drops 99.6% to 96.0%; removing the latent expert drops it to 97.8%.

Limitations

The paper has no limitations section. LIBERO numbers are the best success rate obtained during training. The reported real-world failures are a stuck zipper, a cucumber colliding with the basket, tape slipping from the parallel-jaw gripper, and millimeter-level misalignment in screw insertion when relying on vision alone.

Figures

From Figure 1: Overview. Existing WAMs directly transfer representations of global scene dynamics to action predictio... — From Figure 1: Overview. Existing WAMs directly transfer representations of global scene dynamics to action prediction. LaST0.5 adds Interaction-Centric Latent Reasoning (ICLR), a compact latent space supervised by visual motion and object-affordance cues, between world prediction and robot actions. Right: real-world and simulation results.

From Figure 2: The LaST0.5 framework. (a) A World Expert, Latent Expert and Action Expert within a Mixture-of-Transfo... — From Figure 2: The LaST0.5 framework. (a) A World Expert, Latent Expert and Action Expert within a Mixture-of-Transformers architecture; the Latent Expert distills future frames into compact, interaction-centric representations. (b) Adaptive World-to-Action Transfer (AWAT) predicts progress values and truncates action execution when the value drops. (c) Attention mask.

From Figure 5: LaST0.5 performing the real-world screw fastening and parcel-box packing tasks, with sequences shown f... — From Figure 5: LaST0.5 performing the real-world screw fastening and parcel-box packing tasks, with sequences shown from left to right.

From Figure 6: Out-of-distribution scenario settings: in-domain, position drift, object drift and background drift.

Abstract

World Action Models (WAMs) have recently emerged as a promising paradigm for robotic manipulation, endowing policies with strong capabilities for modeling temporal and physical dynamics. Existing WAMs, encompassing both video-first action prediction and joint video-action modeling, directly transfer global physical representations from world models to action generation. However, global scene representations may underemphasize local robot-object interactions, creating a mismatch between representations optimized for world evolution and those required for manipulation. To bridge this gap, we propose LaST0.5, a novel WAM that introduces an interaction-centric physical representation between world modeling and action generation. Specifically, LaST0.5 learns a compact interaction-centric latent space under the supervision of visual interaction cues between the robot and the environment. This reasoning space is instantiated as a latent expert that bridges the world expert and the action expert within a Mixture-of-Transformers architecture. Furthermore, LaST0.5 performs adaptive world-to-action transfer by jointly predicting future representations and progress values, using progress consistency to adjust action execution and trigger future resampling. Finally, we pretrain LaST0.5 on a large-scale robotic data corpus comprising over 14k hours of data, establishing interaction-centric physical reasoning as a scalable and effective paradigm. LaST0.5 achieves 99.5% on LIBERO, surpasses the previous state of the art on ten evaluated RLBench tasks, and demonstrates robust long-horizon execution and precise dexterous manipulation in real-world experiments.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong