LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning

Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, Peter Stone

Seminal June 2023

TL;DR A simulated benchmark for lifelong robot learning, with 130 procedurally generated manipulation tasks in four suites and 50 human demos per task. In its study, plain sequential finetuning beats the lifelong-learning methods tested at forward transfer, and supervised pretraining on 90 tasks hurts later lifelong learning (AUC 0.32 to 0.22 for ER with ResNet-T).

What they do

LIBERO is a simulation benchmark for lifelong robot manipulation: one language-conditioned policy learns tasks in sequence without access to earlier tasks’ data. Its suites separate declarative knowledge (objects, spatial relations) from procedural knowledge (motions). The authors use it to compare architectures, lifelong-learning algorithms, task orderings, language embeddings and pretraining.

How it works

Task generation. Language templates mined from Ego4D annotations become instructions. A PDDL file then sets the scene, the initial object layout and a goal written as predicates such as On, In or Open. The episode ends when every predicate holds. The pipeline is built on Robosuite.
Suites. LIBERO-Spatial, -Object and -Goal have 10 tasks each and each varies one factor. LIBERO-100 is split into LIBERO-90 (used for pretraining) and LIBERO-Long (10 long-horizon tasks). Every task has 50 Spacemouse teleoperated demos.
Baselines. Three behavior-cloning policies (ResNet-RNN, ResNet-T, ViT-T) with BERT task embeddings and a GMM action head. They are trained with ER, EWC or PackNet, with sequential finetuning (SeqL) and multitask learning as lower and upper bounds. Metrics are forward transfer (FWT), negative backward transfer (NBT) and area under the success curve (AUC).

Results

SeqL has the best FWT on every suite (0.54 on LIBERO-Long with ResNet-T, against 0.48 for ER, 0.22 for PackNet and 0.13 for EWC), but it forgets the most (NBT 0.63).
PackNet has the best AUC on Spatial, Object and Goal (0.63, 0.60, 0.75). ER beats it on LIBERO-Long (0.32 vs 0.25). EWC scores below SeqL (AUC 0.02 vs 0.15 on LIBERO-Long).
Transformer policies beat ResNet-RNN. With ER, ViT-T leads only on LIBERO-Object (AUC 0.57 vs 0.44). BERT, CLIP, GPT-2 and a plain “Task N” embedding give no statistically significant difference (AUC 0.30-0.35).
Pretraining on LIBERO-90 lowers LIBERO-Long performance for all five combinations tested.

Limitations

Simulation only, behavior cloning only, three seeds per setting. The authors leave open better spatial and temporal architectures, algorithms with better forward transfer, and pretraining that actually helps lifelong learning.

Figures

From Figure 1: The four task suites (LIBERO-Spatial, LIBERO-Object and LIBERO-Goal with 10 tasks each, LIBERO-100 wit... — From Figure 1: The four task suites (LIBERO-Spatial, LIBERO-Object and LIBERO-Goal with 10 tasks each, LIBERO-100 with 100) and the five research topics studied on them.

From Figure 2: Task generation, from Ego4D behavior templates to a language instruction, then a PDDL file that sets t... — From Figure 2: Task generation, from Ego4D behavior templates to a language instruction, then a PDDL file that sets the objects and layout, the initial configuration and the goal.

From Figure 5: LIBERO-Long performance of five algorithm and architecture pairs with and without pretraining on LIBER... — From Figure 5: LIBERO-Long performance of five algorithm and architecture pairs with and without pretraining on LIBERO-90, with multitask learning for reference.

Abstract

Lifelong learning offers a promising paradigm of building a generalist agent that learns and adapts over its lifespan. Unlike traditional lifelong learning problems in image and text domains, which primarily involve the transfer of declarative knowledge of entities and concepts, lifelong learning in decision-making (LLDM) also necessitates the transfer of procedural knowledge, such as actions and behaviors. To advance research in LLDM, we introduce LIBERO, a novel benchmark of lifelong learning for robot manipulation. Specifically, LIBERO highlights five key research topics in LLDM: 1) how to efficiently transfer declarative knowledge, procedural knowledge, or the mixture of both; 2) how to design effective policy architectures and 3) effective algorithms for LLDM; 4) the robustness of a lifelong learner with respect to task ordering; and 5) the effect of model pretraining for LLDM. We develop an extendible procedural generation pipeline that can in principle generate infinitely many tasks. For benchmarking purpose, we create four task suites (130 tasks in total) that we use to investigate the above-mentioned research topics. To support sample-efficient learning, we provide high-quality human-teleoperated demonstration data for all tasks. Our extensive experiments present several insightful or even unexpected discoveries: sequential finetuning outperforms existing lifelong learning methods in forward transfer, no single visual encoder architecture excels at all types of knowledge transfer, and naive supervised pretraining can hinder agents' performance in the subsequent LLDM. Check the website at https://libero-project.github.io for the code and the datasets.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong