Learning Human-to-Robot Handovers from Point Clouds

Sammy Christen, Wei Yang, Claudia Pérez-D'Arpino, Otmar Hilliges, Dieter Fox, Yu-Wei Chao

Seminal March 2023

Sim-to-real: a user hands objects to the real Franka robot.

TL;DR A point-cloud handover policy is trained in simulation against replayed human motion, first with the hand held still and planner demonstrations, then fine-tuned while the hand moves. It reaches 75% success on the HandoverSim test scenes (69% with the human moving) and beats GA-DDPG on a real Franka (41/60 vs 21/60 handovers for one subject).

What they do

They learn a human-to-robot handover policy from vision alone. The robot sees segmented hand and object point clouds from its wrist camera and outputs end-effector motions. Training happens in HandoverSim, where a simulated hand replays captured human handovers from DexYCB. Testing covers unseen motions, a second physics engine (Isaac Gym) and a real Franka Panda.

How it works

Two phases per episode. A PointNet++ embedding of the hand and object points feeds a policy that moves the gripper to a pre-grasp pose, plus a grasp predictor that decides when to switch to a scripted grasp (move 8 cm forward, close, retract).
Pretraining with a stationary hand. The robot starts only once the hand stops, so the OMG motion planner can supply demonstrations, using ACRONYM grasps filtered for hand collisions. These are mixed with TD3 exploration under a sparse reward: 1 only for a handover with no collision and no drop.
Fine-tuning with a moving hand. Planners no longer apply here, so a frozen copy of the pretrained policy becomes the behavior-cloning teacher while RL continues.
Real robot. An external Azure Kinect, body tracking and 2D hand segmentation produce the segmented point cloud. It is then transformed into the wrist-camera frame.

Results

HandoverSim test scenes: 75.23% success with the hand held still, versus 50.00% for GA-DDPG (57.18% fine-tuned). With the human moving, 68.75% versus 36.81% (54.86%).
Isaac Gym transfer: 44.21% (hand still) and 39.58% (hand moving). GA-DDPG reaches at most 19.44% and 16.67%.
Real pilot study, 60 controlled poses per subject: 41/60 vs 21/60 (subject 1) and 41/60 vs 33/60 (subject 2). Free-form study with 6 users: 88.9% vs 80.0% success, and 6.40 s vs 7.48 s approach time.
Ablations: without the hand point cloud, success falls to 59.03% and hand contact rises to 24.07%. Without the pre-grasp standoff, success is 52.55%.

Limitations

Small objects that the hand mostly covers are hard to grasp without touching the hand. The grasp predictor is trained offline and sometimes triggers too early. Transparent and dark objects leave gaps in the depth data. Users said GA-DDPG felt safer, because the forward grasp motion brought the gripper close to their fingers. Training takes 72-96 hours on one V100.

Videos

A second participant hands objects to the real robot.

HandoverSim: the policy takes an object from a simulated hand replaying recorded human motion.

Figures

From Figure 1: The policy takes point clouds from a wrist-mounted camera and outputs end-effector actions. It is trai... — From Figure 1: The policy takes point clouds from a wrist-mounted camera and outputs end-effector actions. It is trained against a simulated human in Bullet, tested on unseen handover motions, and transferred to Isaac Gym and to a real robot.

From Figure 2: The perception module turns egocentric depth and segmentation images into a hand/object point cloud, w... — From Figure 2: The perception module turns egocentric depth and segmentation images into a hand/object point cloud, which PointNet++ embeds for the control policy and the grasp predictor. The policy drives the robot to a pre-grasp pose; once the grasp predictor fires, the gripper moves forward, closes and retracts.

From Figure 3: In pretraining the hand is stationary and transitions come from the OMG motion planner and the RL agen... — From Figure 3: In pretraining the hand is stationary and transitions come from the OMG motion planner and the RL agent. In finetuning the human and robot move together, and a frozen copy of the pretrained policy replaces the planner as the expert. Both stages train the actor and critic with behavior cloning, RL and goal-prediction losses.

From Figure 4: Sim-to-real comparison. GA-DDPG goes straight for a grasp, while the proposed method first adjusts the... — From Figure 4: Sim-to-real comparison. GA-DDPG goes straight for a grasp, while the proposed method first adjusts the gripper into a stable grasping pose and keeps hold of the box after the handover.

Abstract

We propose the first framework to learn control policies for vision-based human-to-robot handovers, a critical task for human-robot interaction. While research in Embodied AI has made significant progress in training robot agents in simulated environments, interacting with humans remains challenging due to the difficulties of simulating humans. Fortunately, recent research has developed realistic simulated environments for human-to-robot handovers. Leveraging this result, we introduce a method that is trained with a human-in-the-loop via a two-stage teacher-student framework that uses motion and grasp planning, reinforcement learning, and self-supervision. We show significant performance gains over baselines on a simulation benchmark, sim-to-sim transfer and sim-to-real transfer.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong