Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours
TL;DR A Baxter robot collected 50K self-labeled grasp attempts over 700 hours, enough to train a CNN that scores 18 grasp angles per image patch. It reaches 79.5% accuracy on a held-out set of novel objects and lifts novel objects in 66% of real trials.
What they do
Human-labeled grasp datasets are small and biased: objects can be grasped many ways, and annotators favor parts like handles. Here a Baxter robot instead made random planar grasps on cluttered tables for 700 hours, labeling each attempt with its gripper force sensor: 50K grasps over 150 objects, about 40x prior work. A CNN trained on this data predicts where and at what angle to grasp.
How it works
- Data collection. Background subtraction on a Kinect image finds objects; the robot grasps at a random point and angle, lifts 20 cm and checks the force sensor. Humans only start the robot and scatter objects.
- Formulation. A 380x380 patch around the grasp point goes into the network, which outputs 18 binary graspable scores, one per 10° angle bin. Only the tried angle’s bin receives loss.
- Network. ImageNet-pretrained AlexNet convolutional layers plus two new fully connected layers (18M new parameters).
- Staged learning. The current model picks grasps by importance sampling over 800 scored patches, collecting hard negatives; this data joins the old set and the network is finetuned.
Results
- On 3K held-out grasps of 15 novel objects: 76.9% accuracy after one stage, 79.5% with multi-stage learning, versus 62.1% for the best heuristic, 69.4% for kNN and 73.3% for a linear SVM.
- Gains from more data shrink after about 20K points. ImageNet pretraining lifts accuracy from 64.6% to 76.9%; 2K hard-negative grasps (to 79.3%) help more than 20K random ones.
- Real robot, 150 tries: novel objects are lifted 66% of the time, seen objects in new poses 73%. Clearing 10 cluttered objects takes 26 interactions on average.
Limitations
Planar grasps only. Random trials succeed just 8% of the time, so most data is negative. Failures come from gripper size not matching object width, objects slipping out, and Baxter’s imprecise arm, partly offset by re-ranking top grasps over neighboring patches. Multi-stage gains saturate after 3 stages.
Figures
Abstract
Current learning-based robot grasping approaches exploit human-labeled datasets for training the models. However, there are two problems with such a methodology: (a) since each object can be grasped in multiple ways, manually labeling grasp locations is not a trivial task; (b) human labeling is biased by semantics. While there have been attempts to train robots using trial-and-error experiments, the amount of data used in such experiments remains substantially low and hence makes the learner prone to over-fitting. In this paper, we take the leap of increasing the available training data to 40 times more than prior work, leading to a dataset size of 50K data points collected over 700 hours of robot grasping attempts. This allows us to train a Convolutional Neural Network (CNN) for the task of predicting grasp locations without severe overfitting. In our formulation, we recast the regression problem to an 18-way binary classification over image patches. We also present a multi-stage learning approach where a CNN trained in one stage is used to collect hard negatives in subsequent stages. Our experiments clearly show the benefit of using large-scale datasets (and multi-stage training) for the task of grasping. We also compare to several baselines and show state-of-the-art performance on generalization to unseen objects for grasping.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.