Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach
TL;DR GG-CNN, a 62k-parameter fully convolutional network, predicts a grasp quality, angle and gripper width for every pixel of a depth image in 19 ms, fast enough to re-plan the grasp in a closed loop. It grasps objects that are moved during the attempt with 83% success on adversarial objects and 88% on household objects.
What they do
Deep grasp planners of the time sampled and ranked grasp candidates one at a time. That took from about a second to tens of seconds, so the grasp was executed open-loop and needed a static scene and precise calibration. GG-CNN maps a depth image directly to a grasp at every pixel and is fast enough to run inside a visual servoing loop on a wrist camera.
How it works
- Pixelwise grasp map. For each pixel of a 300x300 depth image the network outputs grasp quality in [0, 1], gripper width, and the grasp angle encoded as sin(2φ) and cos(2φ) to avoid the wrap-around at ±π/2. Grasps are antipodal and top-down.
- Training data. Labels come from the 885 images of the Cornell Grasping Dataset: the centre third of each positive grasp rectangle is painted into the three maps. Augmentation gives 8840 depth images. The chosen network has 62,420 parameters.
- Closed loop. A wrist-mounted RealSense on a Kinova Mico streams depth at 30 Hz. Of the three highest quality peaks, the one nearest the previous target is tracked, and a position-based visual servoing controller drives the gripper pose and finger width toward it. The network takes 6 ms, the full pipeline 19 ms.
Results
- Static single objects: 84% open-loop and 81% closed-loop on 8 adversarial 3D-printed objects, 92% and 91% on 12 household objects.
- Objects moved at least 100 mm and 25° mid-grasp: 83% (66/80) adversarial and 88% (106/120) household, within the 95% confidence bounds of the static results.
- Reproducing Viereck et al.’s clutter experiments without training on clutter: 100% isolated, 87% clutter and 81% dynamic clutter, against 98%, 89% and 77% reported there.
- Under the largest simulated kinematic error, open-loop drops to 38% while closed-loop keeps 68% (adversarial) and 73% (household).
Limitations
The camera gives no accurate depth closer than 150 mm, so the target stops updating about 70 mm from the object and late errors go uncorrected. It also returns no depth on many black or reflective objects. Objects under 15 mm tall cannot be grasped. Comparisons with other papers use different objects and protocols, so they are only indicative.
Videos
Figures
Abstract
This paper presents a real-time, object-independent grasp synthesis method which can be used for closed-loop grasping. Our proposed Generative Grasping Convolutional Neural Network (GG-CNN) predicts the quality and pose of grasps at every pixel. This one-to-one mapping from a depth image overcomes limitations of current deep-learning grasping techniques by avoiding discrete sampling of grasp candidates and long computation times. Additionally, our GG-CNN is orders of magnitude smaller while detecting stable grasps with equivalent performance to current state-of-the-art techniques. The light-weight and single-pass generative nature of our GG-CNN allows for closed-loop control at up to 50Hz, enabling accurate grasping in non-static environments where objects move and in the presence of robot control inaccuracies. In our real-world tests, we achieve an 83% grasp success rate on a set of previously unseen objects with adversarial geometry and 88% on a set of household objects that are moved during the grasp attempt. We also achieve 81% accuracy when grasping in dynamic clutter.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.