MimicPlay: Long-Horizon Imitation Learning by Watching Human Play

Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, Anima Anandkumar

Seminal February 2023

Kitchen task at 4x, with the input view and the generated hand-trajectory plan inset.

TL;DR MimicPlay learns a goal-conditioned latent planner from 10 minutes of human hand play video per scene, then trains a low-level robot policy on its plans from 20-40 teleoperated demos. On long-horizon kitchen tasks it succeeds 70% of the time with 40 demos, against at most 17% for prior imitation-learning baselines.

What they do

Long-horizon imitation learning usually needs hours of teleoperated robot data. MimicPlay splits the problem in two. A high-level planner learns where and what to interact with from human play data: a person freely manipulates the scene with one hand, with no task labels or resets. A low-level controller then learns how to move the robot from a small set of teleoperated demos. They record 10 minutes of human play per environment, which the authors put at roughly 3 hours of robot teleoperation. The evaluation covers 14 tasks in six environments on a Franka arm.

How it works

3D hand trajectories. Two calibrated cameras and an off-the-shelf hand detector recover 3D hand trajectories from the play videos.
Latent planner. ResNet-18 encoders map the current image and a goal image to a latent plan. A 5-mode GMM decoder predicts the future 3D hand trajectory from that plan. A KL loss aligns human-video and robot-video feature distributions, so no paired data is needed.
Plan-guided policy. The frozen planner’s latent plan, wrist-camera features and proprioception are fed to a GPT-style transformer with a GMM action head. It is trained on 20 demos per task (40 in the Kitchen).
Prompting. At test time, successive frames of one human or robot video serve as goal images. The full system runs at 17 Hz.

Results

Kitchen, full long-horizon tasks: 0.47 success with 20 demos and 0.70 with 40. Without human data the scores are 0.23 and 0.40. Baselines (GC-BC, C-BeT, LMP, R3M-BC), given 10 extra minutes of robot demos instead, reach at most 0.07 and 0.17.
Study Desk: 0.55 on trained tasks and 0.47 on unseen subgoal compositions. Without human data: 0.20 and 0.10. Without the GMM: 0.28 and 0.07. Without the KL loss: 0.38 and 0.20.
One multi-task model across Flower, Whiteboard, Sandwich and Cloth scores 0.55, against 0.05 for LMP and 0.15 for R3M. Whiteboard drops from 0.5 with a single-task model to 0.2.
Human-video prompts perform close to robot-video prompts. The robot also replans after human disturbances that never appear in the demos.

Limitations

The planner learns from scene-specific human play, all tasks are on a tabletop, and the authors say the cross-embodiment representation has plenty of room to improve.

Videos

Sandwich task, rotating the ingredient selector and stacking onto the plate.

Study desk task, opening the box while a person disturbs the lid.

Figures

From Figure 1: a human completes a long-horizon task much faster than a teleoperated robot. MimicPlay learns a high-l... — From Figure 1: a human completes a long-horizon task much faster than a teleoperated robot. MimicPlay learns a high-level planner from human play data and a low-level control policy from a small amount of multi-task teleoperated robot demonstrations.

From Figure 2: (a) stage 1 trains a goal-conditioned 3D hand-trajectory model on human play data, which defines the l... — From Figure 2: (a) stage 1 trains a goal-conditioned 3D hand-trajectory model on human play data, which defines the latent plan space; (b) stage 2 trains a low-level robot controller on teleoperation data, conditioned on plans from the frozen planner; (c) at test time a single human or robot video prompt generates the plans that guide the controller.

From Figure 5: the latent plan before a human disturbance, the disturbance itself (red arrow), the re-planned traject... — From Figure 5: the latent plan before a human disturbance, the disturbance itself (red arrow), the re-planned trajectory right after it, and the robot recovering with the updated plan.

Abstract

Imitation learning from human demonstrations is a promising paradigm for teaching robots manipulation skills in the real world. However, learning complex long-horizon tasks often requires an unattainable amount of demonstrations. To reduce the high data requirement, we resort to human play data - video sequences of people freely interacting with the environment using their hands. Even with different morphologies, we hypothesize that human play data contain rich and salient information about physical interactions that can readily facilitate robot policy learning. Motivated by this, we introduce a hierarchical learning framework named MimicPlay that learns latent plans from human play data to guide low-level visuomotor control trained on a small number of teleoperated demonstrations. With systematic evaluations of 14 long-horizon manipulation tasks in the real world, we show that MimicPlay outperforms state-of-the-art imitation learning methods in task success rate, generalization ability, and robustness to disturbances. Code and videos are available at https://mimic-play.github.io

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong