Matt Strong

Matt Strong

CS PhD at Stanford University

← Paper Digest

RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation

Ajay Mandlekar, Yuke Zhu, Animesh Garg, Jonathan Booher, Max Spero, Albert Tung, Julian Gao, John Emmons, Anchit Gupta, Emre Orbay, Silvio Savarese, Li Fei-Fei

Seminal November 2018

A user steers the simulated arm by moving a phone while watching the stream on a laptop.

TL;DR A crowdsourcing platform where remote workers teleoperate a simulated robot arm in 6-DoF by moving an iPhone while watching a video stream in their browser. In 22 hours of use it collected over 2200 successful demonstrations, and using 1000 of them raises sparse-reward RL return on can picking from 0 to 641 out of 1000.

What they do

Imitation learning needs hundreds to thousands of demonstrations. Kinesthetic teaching is tedious, keyboard and 3D-mouse teleoperation give unnatural axis-aligned trajectories, and VR needs hardware that crowd workers rarely own. RoboTurk lets anyone with an ARKit-capable iPhone and a web browser teleoperate a simulated Sawyer arm in 6-DoF from anywhere, so demonstrations can be crowdsourced.

How it works

  • Phone as controller. ARKit tracks the phone pose. The server maps it to a target end-effector pose, solves IK and drives the joints toward it. The phone vibrates when the arm touches an object.
  • Cloud backend. A coordination server starts one MuJoCo teleoperation server per user and opens WebRTC channels to the user’s phone (commands) and browser (rendered video). The user needs no powerful local machine.
  • Using the data. 90% of RL episodes start from a state sampled from a demonstration; distributed PPO then learns from a sparse success reward.

Results

  • User study with 8 students: mean picking time was 89.97 s with the phone, 79.36 s with VR, 112.57 s with a 3D mouse and 151.45 s with a keyboard. Keyboard and 3D mouse were significantly slower; phone and VR were statistically indistinguishable.
  • Cutting bandwidth to 500 Kbps and/or adding 120 ms of one-way delay caused no significant change in completion times. Users in California working against servers in China instead of Oregon took 24-28 s longer on average but still finished the tasks.
  • Pilot dataset: 3224 demonstrations in 22 hours of use, 2218 successful (1171 picking, 1147 assembly), 137.5 hours of trajectories.
  • On simplified can-picking and round-assembly tasks, pure RL gets 0 return. 1000 demonstrations give the best mean return, 641 and 775 out of 1000, against 278 and 381 with a single demonstration.

Limitations

Everything runs in simulation; real robot arms are left to future work. Only ARKit iPhones are supported, and phone tracking is noisier than VR. The user study has 8 participants. The RL results vary widely across 10 seeds, and on round assembly 10 demonstrations (663) beat 100 (575).

Videos

Teleoperation sessions run from the Swiss Alps and from Macau to Stanford.
Several users in different places collecting demonstrations with a phone and a laptop.

Figures

From Figure 1: The RoboTurk pipeline: specify a task, crowdsource many demonstrations, learn a policy with demonstrat...
From Figure 1: The RoboTurk pipeline: specify a task, crowdsource many demonstrations, learn a policy with demonstration-augmented reinforcement learning, then deploy it.
From Figure 2: (a) A coordination server launches a dedicated teleoperation server for each new user and links it to ...
From Figure 2: (a) A coordination server launches a dedicated teleoperation server for each new user and links it to the user's browser and iPhone. (b) The user moves the phone to control the simulated robot and watches the video stream in the browser; each successful demonstration is pushed to cloud storage.
From Figure 5: Completion time distributions of the four user interfaces (phone, VR, 3D mouse, keyboard) on the picki...
From Figure 5: Completion time distributions of the four user interfaces (phone, VR, 3D mouse, keyboard) on the picking task.
Abstract

Imitation Learning has empowered recent advances in learning robotic manipulation tasks by addressing shortcomings of Reinforcement Learning such as exploration and reward specification. However, research in this area has been limited to modest-sized datasets due to the difficulty of collecting large quantities of task demonstrations through existing mechanisms. This work introduces RoboTurk to address this challenge. RoboTurk is a crowdsourcing platform for high quality 6-DoF trajectory based teleoperation through the use of widely available mobile devices (e.g. iPhone). We evaluate RoboTurk on three manipulation tasks of varying timescales (15-120s) and observe that our user interface is statistically similar to special purpose hardware such as virtual reality controllers in terms of task completion times. Furthermore, we observe that poor network conditions, such as low bandwidth and high delay links, do not substantially affect the remote users' ability to perform task demonstrations successfully on RoboTurk. Lastly, we demonstrate the efficacy of RoboTurk through the collection of a pilot dataset; using RoboTurk, we collected 137.5 hours of manipulation data from remote workers, amounting to over 2200 successful task demonstrations in 22 hours of total system usage. We show that the data obtained through RoboTurk enables policy learning on multi-step manipulation tasks with sparse rewards and that using larger quantities of demonstrations during policy learning provides benefits in terms of both learning consistency and final performance. For additional results, videos, and to download our pilot dataset, visit $\href{http://roboturk.stanford.edu/}{\texttt{roboturk.stanford.edu}}$

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.