DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
TL;DR A teleoperated manipulation dataset of 76k demonstrations across 564 scenes and 86 tasks, collected on 18 identical Franka setups at 13 institutions. Co-training a diffusion policy on DROID beats the next best training mix by 22 points of success in distribution and 17 points out of distribution.
What they do
DROID is a teleoperated robot manipulation dataset built for scene diversity: 76k successful demonstrations (350 hours) across 564 scenes in 52 buildings and 86 tasks. Fifty data collectors at 13 institutions in North America, Asia and Europe recorded it over 12 months. Each episode has three stereo camera views, depth, camera calibration and crowdsourced language instructions. The paper measures the dataset’s diversity and tests whether co-training on it improves policies.
How it works
- Shared platform. 18 copies of one setup: a Franka Panda with a Robotiq 2F-85 gripper on a wheeled, height-adjustable desk, two tripod-mounted Zed 2 cameras plus a wrist Zed Mini, and Meta Quest 2 controllers for teleoperation. Actions are recorded at 15 Hz.
- Collection protocol. A GUI picks each episode’s task at random from those the collector listed for the scene and periodically asks for scene changes (base, cameras, lighting, objects). Collectors move on after about 20 minutes per scene.
- Policy test. A language-conditioned diffusion policy (two external cameras, ResNet-50 encoder, U-Net head) trained on 50-150 in-domain demos per task. Training batches are mixed 50/50 with DROID, with Open X-Embodiment (OXE), or not mixed at all.
Results
- 564 scenes, compared with 311 for the whole OXE aggregation and 24 for Bridge V2, plus 1417 distinct camera viewpoints.
- Across 6 tasks in 4 locations (10 rollouts per setting), DROID co-training beats the next best method by 22 points of success in distribution and 17 points out of distribution. On the three-step Cook Lentils task, it is the only method that consistently finishes all steps.
- At an equal size of 7,362 trajectories, a scene-diverse sample of DROID gives better out-of-distribution performance than the 20 scenes with the most demonstrations.
Limitations
All data comes from one robot, the Franka Panda. DROID is only tested as co-training data next to in-domain demonstrations; whether policies work in new scenes without in-domain data is left open. Each setting gets 10 rollouts, and distractors inside the desk made every policy fail. The post-hoc camera calibrations still contain false positives and struggle in cluttered, low-overlap scenes.
Videos
Figures
Abstract
The creation of large, diverse, high-quality robot manipulation datasets is an important stepping stone on the path toward more capable and robust robotic manipulation policies. However, creating such datasets is challenging: collecting robot manipulation data in diverse environments poses logistical and safety challenges and requires substantial investments in hardware and human labour. As a result, even the most general robot manipulation policies today are mostly trained on data collected in a small number of environments with limited scene and task diversity. In this work, we introduce DROID (Distributed Robot Interaction Dataset), a diverse robot manipulation dataset with 76k demonstration trajectories or 350 hours of interaction data, collected across 564 scenes and 84 tasks by 50 data collectors in North America, Asia, and Europe over the course of 12 months. We demonstrate that training with DROID leads to policies with higher performance and improved generalization ability. We open source the full dataset, policy learning code, and a detailed guide for reproducing our robot hardware setup.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.