Octo: An Open-Source Generalist Robot Policy

Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Ria Doshi, Charles Xu, Jianlan Luo, You Liang Tan, Lawrence Yunliang Chen, Pannag Sanketi, Quan Vuong, Ted Xiao, Dorsa Sadigh, Chelsea Finn, Sergey Levine

Seminal May 2024

TL;DR Octo is an open-source transformer policy pretrained on 800k Open X-Embodiment robot episodes that follows language or goal-image commands and finetunes to new sensors, action spaces and robots. Finetuned on about 100 demos per task, it averages 72% success across six real-robot setups, versus 20% for the best baseline.

What they do

Octo is a generalist robot policy pretrained on 800k demonstrations from 25 Open X-Embodiment datasets, released fully open source: checkpoints, finetuning and pretraining code, and data loaders. It takes a language instruction or a goal image, controls several robots out of the box, and adapts to new cameras, sensors, action spaces or robots by finetuning on about 100 demonstrations.

How it works

Tokenizers. Language goes through a frozen t5-base encoder; images go through a shallow CNN and are split into 16x16 patches. Missing inputs are masked out, so datasets with different cameras and labels train one model.
Backbone and head. A ViT-style transformer (27M Small, 93M Base) with block-wise causal attention over two frames of history. Learned readout tokens read the sequence without influencing it, and a small diffusion head turns them into a chunk of future actions.
Finetuning. A new input or action space only needs a new encoder or head. The full model is finetuned for 50k steps with one shared recipe, about 5 hours on a 24 GB A5000.

Results

Zero-shot on WidowX, UR5 and the RT-1 robot: on average 29% higher success than RT-1-X, and similar to the 55B-parameter RT-2-X on WidowX and the RT-1 robot. On WidowX, goal images give 25% higher success than language.
Finetuning with ~100 demos: 72% average over six setups, against 20% for a from-scratch ResNet+Transformer and 15% for VC-1. The setups include force-torque input, joint position control, an unseen ViperX arm and a bimanual ALOHA.
Ablations (WidowX, Octo-Small at 83%): the 11-dataset RT-X mix drops to 60%, Bridge data alone to 43%, a ResNet-50 encoder to 70%, an MSE head to 35% and discretized actions to 18%. Zero-shot success grows from 10M to 93M parameters.

Limitations

Octo handles wrist cameras poorly: finetuning was often stronger with the third-person camera alone. Language conditioning trails goal images. The authors attribute both to the data, where 27% of episodes have wrist images and 56% have language. Zero-shot success falls to 40% in new environments and 5% on unseen skills. Training uses only optimal demonstrations, and only single- and dual-arm manipulators were tested.

Videos

Figures

From Figure 1: Octo, a transformer policy pretrained on 800k robot episodes, takes language or goal-image tasks and w... — From Figure 1: Octo, a transformer policy pretrained on 800k robot episodes, takes language or goal-image tasks and wrist or third-person cameras, controls several robots out of the box, and finetunes to new observation and action spaces.

From Figure 1: Model architecture. Task tokens (green, from a pretrained language model) and observation tokens (blue... — From Figure 1: Model architecture. Task tokens (green, from a pretrained language model) and observation tokens (blue, from a lightweight CNN) feed the transformer, whose readout tokens (purple) go to action heads. For finetuning, new observations or action heads (dashed) are added without modifying pretrained parameters.

From Figure 4: Zero-shot success with language-specified tasks. Octo outperforms RT-1-X on WidowX, UR5 and the RT-1 r... — From Figure 4: Zero-shot success with language-specified tasks. Octo outperforms RT-1-X on WidowX, UR5 and the RT-1 robot, and performs similarly to RT-2-X on the WidowX and RT-1 robot tasks.

Abstract

Large policies pretrained on diverse robot datasets have the potential to transform robotic learning: instead of training new policies from scratch, such generalist robot policies may be finetuned with only a little in-domain data, yet generalize broadly. However, to be widely applicable across a range of robotic learning scenarios, environments, and tasks, such policies need to handle diverse sensors and action spaces, accommodate a variety of commonly used robotic platforms, and finetune readily and efficiently to new domains. In this work, we aim to lay the groundwork for developing open-source, widely applicable, generalist policies for robotic manipulation. As a first step, we introduce Octo, a large transformer-based policy trained on 800k trajectories from the Open X-Embodiment dataset, the largest robot manipulation dataset to date. It can be instructed via language commands or goal images and can be effectively finetuned to robot setups with new sensory inputs and action spaces within a few hours on standard consumer GPUs. In experiments across 9 robotic platforms, we demonstrate that Octo serves as a versatile policy initialization that can be effectively finetuned to new observation and action spaces. We also perform detailed ablations of design decisions for the Octo model, from architecture to training data, to guide future research on building generalist robot models.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong