Grasping in the Wild: Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations

Shuran Song, Andy Zeng, Johnny Lee, Thomas Funkhouser

Seminal December 2019

TL;DR People record 12 hours of grasping demonstrations in everyday places with a $600 handheld grabber that carries a wrist RGB-D camera, and these pretrain a closed-loop 6DoF grasping Q-function that scores rendered future views. After finetuning on a UR5, it grasps novel objects 92% of the time in static scenes and 88% while the objects are being moved.

What they do

Most learned grasping is top-down or open-loop, because a 6DoF closed-loop policy needs too much trial-and-error data on the robot. The authors had people do everyday pick-and-place tasks with a handheld grabber that mimics a robot gripper and its wrist camera. The recordings pretrain a closed-loop 6DoF grasping Q-function, which is then finetuned on a real UR5.

How it works

Demonstrations. A plastic grabber with a servo-driven binary close and a RealSense D415 25 cm from the fingertips, about $600 in parts. Eight participants recorded 12 h of RGB-D video in apartments, kitchens, offices and warehouses. Gripper poses come from frame-to-frame tracking (SIFT and RANSAC, refined with ICP), which gives 7,797 picking attempts.
Action-views. On the robot, wrist-camera frames are fused into a TSDF with 5 mm voxels at 30 FPS. For each of 35 candidate 6DoF moves, the system ray-casts the view the camera would see after the move.
Q-function. One encoder takes the current view and one takes each candidate view. Their features feed a network that predicts pixel-wise Q-values, and the robot executes the best move.
Training. In pretraining, the rendered view closest to the demonstrated pose is the positive example. Views ranked below the top 4 are negatives, sampled at a 1:5 ratio. Finetuning is off-policy Q-learning with ε-greedy exploration on the robot.

Results

Success before/after finetuning: Tabletop 76/92%, Bin 66/82%, Wall 78/89%, randomly placed and tilted bin 62/76%.
Static scenes: 92 ± 5%, against 87-90% for GG-CNN, Viereck et al. and Zeng et al. Dynamic scenes (the pile is moved by hand during each grasp): 88 ± 8%, against 81% and 77%.
On Tabletop, pretraining on demonstrations learns faster and ends higher than training by trial and error on the robot alone.
0.18 s per action step on a GTX 2080Ti.

Limitations

View-based rendering models motion and passive observation but not contact physics. The best numbers need trial-and-error finetuning on the robot for each scene setting. The TSDF relies on the UR5’s accurate end-effector poses for camera extrinsics. The placing trajectories in the dataset go unused.

Figures

From Figure 1: A person collects grasping demonstrations with the low-cost handheld device (top row), and the learned... — From Figure 1: A person collects grasping demonstrations with the low-cost handheld device (top row), and the learned 6DoF closed-loop policy picks from a bin on a real robot (bottom).

From Figure 2: The handheld device, a plastic grabber with an RGB-D camera and a servo that opens and closes the fing... — From Figure 2: The handheld device, a plastic grabber with an RGB-D camera and a servo that opens and closes the fingers (left), mirrors the robot's UR5, RG2 gripper and wrist camera (right).

From Figure 4: Left to right: the current camera view, the TSDF-fused scene, action-views rendered for candidate move... — From Figure 4: Left to right: the current camera view, the TSDF-fused scene, action-views rendered for candidate moves, and the selection network that predicts dense Q-values for each action-view pair.

From Figure 7: Tabletop grasping success versus robot training steps for direct trial and error, pretraining on demon... — From Figure 7: Tabletop grasping success versus robot training steps for direct trial and error, pretraining on demonstrations plus finetuning, and pretraining only.

Abstract

Intelligent manipulation benefits from the capacity to flexibly control an end-effector with high degrees of freedom (DoF) and dynamically react to the environment. However, due to the challenges of collecting effective training data and learning efficiently, most grasping algorithms today are limited to top-down movements and open-loop execution. In this work, we propose a new low-cost hardware interface for collecting grasping demonstrations by people in diverse environments. Leveraging this data, we show that it is possible to train a robust end-to-end 6DoF closed-loop grasping model with reinforcement learning that transfers to real robots. A key aspect of our grasping model is that it uses "action-view" based rendering to simulate future states with respect to different possible actions. By evaluating these states using a learned value function (Q-function), our method is able to better select corresponding actions that maximize total rewards (i.e., grasping success). Our final grasping system is able to achieve reliable 6DoF closed-loop grasping of novel objects across various scene configurations, as well as dynamic scenes with moving objects.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong