Matt Strong

Matt Strong

CS PhD at Stanford University

← Paper Digest

Grasp Pose Detection in Point Clouds

Andreas ten Pas, Marcus Gualtieri, Kate Saenko, Robert Platt

Seminal June 2017

TL;DR Sample 6-DOF parallel-jaw grasp candidates on any visible surface of a point cloud and score each with a small CNN over multi-view projections of the gripper's closing region. A Baxter clearing dense clutter of novel objects succeeds on 93% of 288 grasp attempts.

What they do

Grasp pose detection finds 6-DOF parallel-jaw grasps directly in a point cloud, with no object segmentation, CAD model or pose estimate. The paper combines a candidate sampler, a new grasp descriptor and a CNN classifier, measures what each piece contributes, and evaluates the full system on a dense clutter benchmark with novel household objects.

How it works

  • Sampling. Pick points in the region of interest, build a local frame from smoothed surface normals and curvature axes, grid-search hand offsets and rotations, and push the hand forward until contact. Keep hands that are collision-free and have cloud points between the fingers.
  • Descriptor. Voxelize the closing region and project it along three hand axes into heightmaps of occupied and occluded space plus surface normals, 15 channels in all. A cheaper 3-channel normals-only version runs on the robot.
  • Classifier. A LeNet-style four-layer CNN trained on 1.5M exemplars from 55 BigBird objects (real sensor clouds paired with meshes), labelled with a frictionless antipodal check and optionally pretrained on 3DNET CAD data.
  • Selection. Prune grasps outside the 3-7 cm gripper width or without collision-free IK, then prefer high, top-down grasps near a nominal arm configuration.

Results

  • The 15-channel descriptor reaches 89% recall at 99% precision, against 82% without the occlusion channels and 19% for the Herzog/Kappler descriptor.
  • Prior object knowledge helps: accuracy rises from 92% (all objects) to 93.5% (same category) and 97% (same instance) for cylinders, and from 90% to 93% and 95% for boxes.
  • On Baxter, clearing piles of 10 novel objects: 93% success (268/288) with an active SLAM point cloud, 84% with two fixed sensors, 75% without the selection heuristics and 53% without the classifier.
  • Using detected grasps as object proposals gives 85.4% object classification accuracy over 1197 proposals in clutter.

Limitations

Without segmentation the detector can return grasps that span several objects, which the authors expect to matter more for wider hands. Grasping a specific object relies on the added object detector, whose 85% accuracy in clutter must improve before real-world use. Only two-finger hands are handled.

Figures

From Figure 1: the robot grasping in dense clutter, the setting the method targets.
From Figure 1: the robot grasping in dense clutter, the setting the method targets.
From Figure 3: grippers placed at three sampled grasp candidates on a partial point cloud of a soup can.
From Figure 3: grippers placed at three sampled grasp candidates on a partial point cloud of a soup can.
From Figure 5: the local voxel grid of one candidate, with observed points in magenta, occluded points in blue and th...
From Figure 5: the local voxel grid of one candidate, with observed points in magenta, occluded points in blue and the three projection directions as arrows.
From Figure 8: classification accuracy during training for four grasp representations; the 15-channel descriptor (gre...
From Figure 8: classification accuracy during training for four grasp representations; the 15-channel descriptor (green) is best and the Herzog/Kappler descriptor (cyan) is worst.
Abstract

Recently, a number of grasp detection methods have been proposed that can be used to localize robotic grasp configurations directly from sensor data without estimating object pose. The underlying idea is to treat grasp perception analogously to object detection in computer vision. These methods take as input a noisy and partially occluded RGBD image or point cloud and produce as output pose estimates of viable grasps, without assuming a known CAD model of the object. Although these methods generalize grasp knowledge to new objects well, they have not yet been demonstrated to be reliable enough for wide use. Many grasp detection methods achieve grasp success rates (grasp successes as a fraction of the total number of grasp attempts) between 75% and 95% for novel objects presented in isolation or in light clutter. Not only are these success rates too low for practical grasping applications, but the light clutter scenarios that are evaluated often do not reflect the realities of real world grasping. This paper proposes a number of innovations that together result in a significant improvement in grasp detection performance. The specific improvement in performance due to each of our contributions is quantitatively measured either in simulation or on robotic hardware. Ultimately, we report a series of robotic experiments that average a 93% end-to-end grasp success rate for novel objects presented in dense clutter.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.