SLA: Scaling Spatial-Language-Action Model via Continual Pre-training

Fuhao Li, Wenxuan Song, Qizhou Huang, Ziyang Zhou, Jingbo Wang, Junwu E, Pengwei Wang, Yan Wang, Shunbo Zhou, Haoang Li

November 2026

TL;DR Continue pre-training π0.5 on 1.7M episodes of public robot data while aligning its visual tokens with features from a frozen 3D model, then keep that alignment during task finetuning. SLA averages 83.8% success across eight real-robot conditions, 24 points above π0.5, with the inference architecture unchanged.

What they do

VLA models inherit their vision from 2D image-text pretraining, and a depth probe on their frozen visual tokens recovers no meaningful 3D structure. SLA adds spatial supervision while continuing to pre-train π0.5 on about 1.7M episodes (14,330 h) from six public datasets, then keeps it during task post-training. The 3D teacher is used only in training, so the policy keeps π0.5’s inference architecture and needs no depth input.

How it works

Spatial alignment. Visual tokens are projected by an MLP and pulled, with a cosine loss added to the flow-matching action loss, toward features of a frozen OmniVGGT model at the same image location. OmniVGGT also uses camera calibration and depth where a dataset provides them.
Adaptive layer selection. Learned per-token, per-channel gates mix visual tokens from all 18 backbone layers instead of aligning one hand-picked layer.
Unified data. Actions are delta end-effector poses in one robot-centered frame, with rotations composed on SO(3) and stored as rotation vectors; repetitive scripted episodes are subsampled.
State dropout. Robot-state tokens are randomly dropped from training samples (25% works best) so the policy has to use vision.

Results

Real robots (bimanual Piper and Franka, 50 scored objects per condition): 83.8% average over eight conditions, vs 68.0% for post-training-only Spatial Forcing and 59.8% for π0.5. Grasping fruit a person moves: 76% vs 42% for π0.5. Franka with 10 demonstrations (plus 100 Piper ones): 78% vs 48%.
RoboDojo-Real, one policy for three arms: 33.9% success vs 12.8% for π0.5.
RoboTwin 2.0 Clean2Random: 72.3% vs 46.0% for π0.5. With alignment in only one stage it reaches 63.2% (pre-training) or 57.1% (post-training); without state dropout the RoboTwin average falls from 79.2% to 38.1%.

Limitations

The paper has no limitations section. The corpus excludes dexterous hands and mobile-base motion, so SLA covers two-finger arms only. On RoboDojo-Real, pen capping and mug hanging on Piper X never complete; for RoboDojo-Sim the authors name randomized layouts, multi-stage completion and tasks without demonstrations as the main open challenges. Continual pre-training takes about 110 h on 48 H100 GPUs.

Figures

From Figure 1: SLA unifies heterogeneous robot data, aligns intermediate visual embeddings with frozen 3D features du... — From Figure 1: SLA unifies heterogeneous robot data, aligns intermediate visual embeddings with frozen 3D features during training, and improves benchmark and real-world manipulation performance.

From Figure 2: Depth probing of VLA visual embeddings, in simulation and in the real world. Embeddings learned only f... — From Figure 2: Depth probing of VLA visual embeddings, in simulation and in the real world. Embeddings learned only from 2D images give no meaningful depth structure; aligned embeddings recover it.

From Figure 6: Real-robot tabletop tasks: grasp a water bottle (E), a watch from a table at a new height (F) or from ... — From Figure 6: Real-robot tabletop tasks: grasp a water bottle (E), a watch from a table at a new height (F) or from a rising and lowering table (G), and unseen black tape next to a carrot distractor (H). One policy, trained on 100 demonstrations of E, runs all four.

From Figure 4: Continual pre-training data scaling on RoboTwin Clean2Random. SLA alignment is switched on or off sepa... — From Figure 4: Continual pre-training data scaling on RoboTwin Clean2Random. SLA alignment is switched on or off separately in continual pre-training and post-training; all curves start from the π0.5 baseline.

Abstract

Reliable robotic manipulation requires both semantic understanding and precise control in three-dimensional space. However, vision-language-action (VLA) foundations largely inherit visual representations from 2D vision-language pre-training, while spatial supervision is often introduced only during task-specific adaptation. Developing spatial understanding as a transferable foundation-model capability therefore calls for learning from broader robot experience. We present SLA, a Spatial-Language-Action Model that scales implicit spatial alignment to continual pre-training and maintains it during downstream post-training. Initialized from a pretrained pi05 policy, SLA jointly optimizes action prediction and alignment between intermediate visual embeddings and spatial representations from a frozen 3D perception model. We curate and unify heterogeneous robot datasets to support spatial learning across diverse embodiments, tasks, and scenes. To accommodate this diversity, we introduce feature-dependent gates that adaptively aggregate visual embeddings from multiple VLA layers for spatial supervision. The teacher further exploits partially available depth and camera calibration alongside RGB observations, enabling effective use of heterogeneous geometric annotations. Probabilistic state dropout encourages reliance on visual cues and mitigates overfitting to demonstration trajectories. All auxiliary components are used only during training, allowing direct reuse of pretrained VLA weights and preserving the original inference architecture.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong