BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning

Eric Jang, Alex Irpan, Mohi Khansari, Daniel Kappler, Frederik Ebert, Corey Lynch, Sergey Levine, Chelsea Finn

Seminal February 2022

TL;DR BC-Z trains one behavior-cloning policy on 25,877 robot demonstrations of 100 manipulation tasks, conditioned on a sentence or a human video. Given only a new sentence, it succeeds at least sometimes on 24 of 29 unseen tasks, averaging 44% success on those 24.

What they do

BC-Z asks whether plain behavior cloning, scaled to many tasks, can do new manipulation tasks without any robot demos of them. Seven operators on 12 robots collected 25,877 demonstrations (125 hours) of 100 tasks, plus 18,726 human videos of the same tasks. The policy is conditioned on a language command or a human video and is tested on 29 held-out tasks that pair objects never seen together in training.

How it works

Data with interventions. VR teleoperation first gives 11,108 expert demos. After that, the current policy runs while the operator takes over when it is about to fail (HG-DAgger), adding 14,769 episodes over 16 iterations.
Task encoders. Language goes through a frozen pretrained sentence encoder (512-dim). A ResNet-18 video encoder is trained end-to-end with the policy plus a cosine loss that pulls each video’s embedding toward its task’s language embedding.
Policy. A ResNet18 with FiLM task conditioning maps one RGB image to delta end-effector pose and gripper angle at 10 Hz. Actions are state differences to an adaptively chosen future step, which avoids tiny, dithering actions.

Results

Zero-shot from language: 32% average over the 29 held-out tasks, nonzero on 24, and 44% on those 24. Video conditioning reaches 4% overall, nonzero on 9 tasks.
On training tasks, one-hot, language and video conditioning get 42%, 40% and 24%, so language-conditioned performance is limited by control more than by the embedding.
On “place the bottle in the ceramic bowl”, the multi-task policy gets 52%, while a single-task policy trained on that task’s 1000 demos gets 5%. Without adaptive state differences, success falls to 3%. At equal data, a 50/50 expert and HG-DAgger mix roughly doubles success (53% vs 27%).

Limitations

Success on new tasks varies widely. Commands follow a simple “(verb) (noun)” form, and video conditioning lags far behind language. Most failures are last-centimeter errors: not closing the gripper, not releasing an object, or narrowly missing the target.

Videos

Figures

From Figure 1: Overview of BC-Z. Shared-autonomy data collection builds a dataset of 25,877 episodes over 100 manipul... — From Figure 1: Overview of BC-Z. Shared-autonomy data collection builds a dataset of 25,877 episodes over 100 manipulation tasks. A 7-DoF multi-task policy trained on it, conditioned on a language string or a human video, then performs unseen tasks.

From Figure 3: BC-Z network architecture. A ResNet18 encodes the head-camera RGB image and two-layer MLP heads predic... — From Figure 3: BC-Z network architecture. A ResNet18 encodes the head-camera RGB image and two-layer MLP heads predict delta XYZ, delta axis-angle and gripper angle. FiLM layers condition the network on a task embedding computed from a language command or a human video.

From Figure 4: BC-Z performing held-out tasks: placing the apple in the paper cup and placing the bottle on the table... — From Figure 4: BC-Z performing held-out tasks: placing the apple in the paper cup and placing the bottle on the table surface.

Abstract

In this paper, we study the problem of enabling a vision-based robotic manipulation system to generalize to novel tasks, a long-standing challenge in robot learning. We approach the challenge from an imitation learning perspective, aiming to study how scaling and broadening the data collected can facilitate such generalization. To that end, we develop an interactive and flexible imitation learning system that can learn from both demonstrations and interventions and can be conditioned on different forms of information that convey the task, including pre-trained embeddings of natural language or videos of humans performing the task. When scaling data collection on a real robot to more than 100 distinct tasks, we find that this system can perform 24 unseen manipulation tasks with an average success rate of 44%, without any robot demonstrations for those tasks.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong