DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation
TL;DR A faster, more robust optimizer for force-closure grasps produces DexGraspNet, 1.32 million simulation-validated ShadowHand grasps on 5355 objects. Training the DDG grasp network on it instead of the earlier 6.9K-grasp dataset raises simulated success from 57.4% to 67.5%.
What they do
Dexterous grasping lacks large training datasets: the only robotic one before this, DDGdata, has 6.9K ShadowHand grasps on 565 objects, generated with GraspIt!. The authors speed up and harden an existing differentiable force-closure grasp optimizer and use it to build DexGraspNet: 1.32M ShadowHand grasps on 5355 objects, with more than 200 grasps per object.
How it works
- Objects. 3980 ShapeNet models from 133 categories plus 1375 scanned objects, each at 5 scales, remeshed into manifolds and convex-decomposed for simulation.
- Initialization. Hands start open, placed on the object’s inflated convex hull with the palm facing the object, then jittered. This raises yield and makes grasps look more natural.
- Energy and optimizer. A reverse penetration energy (object points against the hand mesh) works on meshes with no thickness. New terms penalize self-penetration and out-of-limit joints. Plain gradient descent converges in under 6000 steps.
- Validation. Each grasp must hold the object in Isaac Gym under all 6 axis-aligned gravity directions and penetrate less than 0.1 cm.
Results
- 10000 grasps take 74 min on an A100 with about 18% valid, versus about 3% valid for the original method. The whole dataset took 950 GPU hours, against an estimated 50000 for the original.
- Against DDGdata, mean Q1 rises from 0.0712 to 0.1145 and mean joint-angle entropy from 4.246 to 5.962.
- Trained on DexGraspNet, both benchmarked methods do better on every test set and metric. Tested on DDGdata, DDG reaches 75.9% success versus 56.4%, and GraspTTA 39.3% versus 23.7%.
Limitations
Optimization pulls every contact toward the object, so grasps are contact-rich power grasps and precision grasps hardly appear. The method has no semantic guidance, so it cannot target functional grasps such as holding a mug by its handle. No existing method trained on the data achieves both its diversity and a reasonable success rate.
Figures
Abstract
Robotic dexterous grasping is the first step to enable human-like dexterous object manipulation and thus a crucial robotic technology. However, dexterous grasping is much more under-explored than object grasping with parallel grippers, partially due to the lack of a large-scale dataset. In this work, we present a large-scale robotic dexterous grasp dataset, DexGraspNet, generated by our proposed highly efficient synthesis method that can be generally applied to any dexterous hand. Our method leverages a deeply accelerated differentiable force closure estimator and thus can efficiently and robustly synthesize stable and diverse grasps on a large scale. We choose ShadowHand and generate 1.32 million grasps for 5355 objects, covering more than 133 object categories and containing more than 200 diverse grasps for each object instance, with all grasps having been validated by the Isaac Gym simulator. Compared to the previous dataset from Liu et al. generated by GraspIt!, our dataset has not only more objects and grasps, but also higher diversity and quality. Via performing cross-dataset experiments, we show that training several algorithms of dexterous grasp synthesis on our dataset significantly outperforms training on the previous one. To access our data and code, including code for human and Allegro grasp synthesis, please visit our project page: https://pku-epic.github.io/DexGraspNet/.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.