RLBench: The Robot Learning Benchmark & Learning Environment

Stephen James, Zicong Ma, David Rovick Arrojo, Andrew J. Davison

Seminal September 2019

TL;DR RLBench is a simulated benchmark of 100 hand-designed manipulation tasks for a Franka Panda, each with unlimited motion-planned demonstrations and text descriptions. The paper proposes a few-shot challenge on 10 held-out tasks but reports no baseline results.

What they do

Manipulation methods are usually tested on tasks each group builds for itself, or on control benchmarks such as OpenAI Gym and the DeepMind Control Suite, whose tasks the authors call toy problems. RLBench is a shared benchmark and learning environment for vision-guided manipulation, built on V-REP and PyRep: 100 unique tasks, from reaching and door opening to multi-stage tasks such as opening an oven and placing a tray in it.

How it works

Scene. Every task uses the same Franka Panda on a wooden table. Observations are RGB, depth and segmentation masks from an over-the-shoulder stereo camera and an eye-in-hand camera, plus joint angles, velocities, torques and end-effector pose. No object starts in the gripper, so tool tasks must first grasp the tool.
Tasks, variations, episodes. Each task has one or more variations (e.g. a different target colour), each with text descriptions; unlimited episodes with new object positions can be drawn from each variation.
Interface. A sparse +1 reward on completion only. Action spaces cover absolute or delta joint velocities, positions and torques, and end-effector velocities or poses.
Demos and task builder. A task is a V-REP model file holding the scene and demo waypoints plus a short Python file for variations and success criteria. OMPL motion planning through the waypoints generates demos on demand, and a validator checks that planning rarely fails before a task is contributed.

Results

The paper runs no experiments or baselines. What it defines:

A few-shot challenge (v1.0): 10% of the 100 tasks are held out as meta-test; methods train on the rest and report 1-, 5- and 20-shot success on new episodes of the unseen tasks.
Suggested uses beyond few-shot: RL with demonstrations, imitation learning, multi-task learning, sim-to-real (the Panda can be swapped for another arm with one line of code) and SLAM for manipulation.

Limitations

Simulation only: the authors do not claim photorealistic rendering or generally realistic physics, though a domain-randomised rendering option is included. No baseline numbers are reported for any task.

Videos

Figures

From Figure 1: 24 of the 100 tasks, including stacking six coloured blocks in a pyramid (top left), inserting a shape... — From Figure 1: 24 of the 100 tasks, including stacking six coloured blocks in a pyramid (top left), inserting a shape onto a peg (top right), setting up a checkers board (bottom left) and watering a plant (bottom right).

From Figure 2: The shared V-REP scene: a Franka Panda on a wooden table, watched by an over-the-shoulder stereo camer... — From Figure 2: The shared V-REP scene: a Franka Panda on a wooden table, watched by an over-the-shoulder stereo camera and an eye-in-hand monocular camera.

From Figure 3: RGB, depth and segmentation-mask observations from the over-the-shoulder stereo and eye-in-hand cameras.

From Figure 4: Task, variation and episode for stack_blocks. Variations change the target blocks (how many, which col... — From Figure 4: Task, variation and episode for stack_blocks. Variations change the target blocks (how many, which colour) and each has text descriptions; episodes change object positions.

Abstract

We present a challenging new benchmark and learning-environment for robot learning: RLBench. The benchmark features 100 completely unique, hand-designed tasks ranging in difficulty, from simple target reaching and door opening, to longer multi-stage tasks, such as opening an oven and placing a tray in it. We provide an array of both proprioceptive observations and visual observations, which include rgb, depth, and segmentation masks from an over-the-shoulder stereo camera and an eye-in-hand monocular camera. Uniquely, each task comes with an infinite supply of demos through the use of motion planners operating on a series of waypoints given during task creation time; enabling an exciting flurry of demonstration-based learning. RLBench has been designed with scalability in mind; new tasks, along with their motion-planned demos, can be easily created and then verified by a series of tools, allowing users to submit their own tasks to the RLBench task repository. This large-scale benchmark aims to accelerate progress in a number of vision-guided manipulation research areas, including: reinforcement learning, imitation learning, multi-task learning, geometric computer vision, and in particular, few-shot learning. With the benchmark's breadth of tasks and demonstrations, we propose the first large-scale few-shot challenge in robotics. We hope that the scale and diversity of RLBench offers unparalleled research opportunities in the robot learning community and beyond.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong