CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation

Choo Sin Wai, Dake Zhong, Jingye Zhang, Zhifang Liu, Siyao Xiao, Zirong Ding, Yuhong Zhang, Kangjie Chen, Dongbin Zhang, Guorui Song, Guocun Wang, Peng Jiao, Haoqian Wang

November 2026

TL;DR CIALLO is an action codec pretrained on eight embodiments that maps actions, visual changes and language into one shared intent space, and adapts to a new robot from a single recording of at most 10 seconds. A Qwen-based policy on top reaches 76.7% success on real SO-101 and Piper arms with 20 demonstrations per task, against 70.0% for fully finetuned pi0.

What they do

Robots record actions in different coordinates, dimensions and control rates, so demonstrations from one robot do not transfer directly to another. CIALLO is an action codec that separates a shared, short-horizon behavioral intent from the embodiment-specific controls that realize it. A policy then only has to predict intent; the codec turns it into actions for the robot at hand.

How it works

Three paths, one space. An action encoder, a vision encoder (current and future frame through a frozen DINOv2) and a text codec (frozen BERT, grounded in the current frame) each produce an intent of two 512-dimensional tokens. Contrastive losses align the three, and one shared decoder must reconstruct the action chunk from each.
Embodiment conditioning. Each robot has a 128-dimensional essential vector plus arm and axis embeddings, injected through conditional normalization in the encoders and the decoder.
Calibrating a new robot. From one recording of at most 10 seconds, only those embodiment parameters are fitted (about 30 minutes on one GPU); the shared weights stay fixed.
Policy. Qwen-CIALLO trains a Qwen3.5-4B backbone and a projector to predict intent, supervised by the frozen action encoder and by decoded actions.

Results

Pretraining uses 125,434 episodes from eight embodiments. On held-out samples, action intent retrieves its paired visual intent with 96.68% top-1 accuracy and its paired language intent with 91.94%.
Real robots (SO-101 and single-arm Piper, both absent from pretraining; 20 demonstrations and 20 trials per task): 76.7% average success vs 70.0% for fully finetuned pi0. The gain comes from the wooden-fish task (70% vs 35% and 40% vs 20%); pi0 is ahead on water-bottle placement.
Simulation, with the action decoder also finetuned: 97.3% on LIBERO after 10k steps, 80.8% on LIBERO-Plus with the same checkpoint, and 52.7% on RoboTwin 2.0 with one policy for all 50 tasks.

Limitations

The real-robot study is small: two arms and three tasks, and the Piper appears in pretraining as a simulated dual-arm setup. LIBERO results are from a single seed, and the authors note that the comparison of finetuning steps does not isolate the effect of the codec. No generalist policy has been pretrained to predict CIALLO intents; that and larger, more diverse pretraining data are left to future work.

Figures

From Figure 1: Overview of CIALLO. (A) During pretraining, three encoders map an action chunk with proprioception, a ... — From Figure 1: Overview of CIALLO. (A) During pretraining, three encoders map an action chunk with proprioception, a pair of current and future frames, and an instruction grounded in the current frame into a shared intent space; a shared decoder reconstructs the demonstrated actions from each intent. (B) Qwen-CIALLO predicts intent from observations and instructions, with the frozen action encoder as a teacher. (C) For a new robot, only its essential vector and arm and axis embeddings are calibrated, then the policy is trained with the codec frozen.

From Figure 3: Vision and text paths. The vision encoder compares current and future observations with a frozen DINOv... — From Figure 3: Vision and text paths. The vision encoder compares current and future observations with a frozen DINOv2 backbone and pools tokens from multiple cameras into chunk-level intent. The text codec compresses the instruction into a task-level intent and grounds it using features from the current frame.

From Figure 5: Intent geometry at the final pretraining checkpoint, shown as a three-dimensional t-SNE embedding. Col... — From Figure 5: Intent geometry at the final pretraining checkpoint, shown as a three-dimensional t-SNE embedding. Color identifies the embodiment and marker shape identifies the input modality.

From Figure 6: A front camera frame from a demonstration of the pour beer task on the SO-101, one of the three real r... — From Figure 6: A front camera frame from a demonstration of the pour beer task on the SO-101, one of the three real robot tasks.

Abstract

Robot demonstrations contain reusable manipulation knowledge, but transferring this knowledge across embodiments requires representations that bridge differences in control interfaces. Action reconstruction alone does not ensure that similar behaviors across robots share a representation. To address this gap, we introduce CIALLO, an action codec that aligns actions, visual changes, and observation-grounded language in a shared space of short-horizon behavioral intent. An embodiment-conditioned decoder maps this intent back to robot-specific actions, allowing a shared representation to support different control interfaces. After pretraining on eight embodiments, CIALLO adapts to a new robot from one recording of at most 10 seconds by updating only its embodiment embeddings. The calibrated codec then remains frozen while a policy learns to predict intent from observations and instructions. We instantiate this policy with Qwen3.5-4B to form Qwen-CIALLO, whose backbone requires no large-scale robot-action pretraining before policy finetuning. With 20 demonstrations per task, Qwen-CIALLO achieves 76.7\% average success across real SO-101 and single-arm Piper robots, compared with 70.0\% for fully finetuned $π_0$. In simulation, we additionally finetune the action decoder, achieving 97.3\% success on LIBERO and 52.7\% on RoboTwin 2.0. The LIBERO policy also achieves 80.8\% on LIBERO-Plus without further finetuning. Together, these results support a shared action codec as a transferable control interface that separates embodiment calibration from task learning.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong