Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?
TL;DR Pretrain one dexterous-hand RL policy by playing with a thousand random objects in simulation, then finetune it on sparse-reward assembly. Play2Perfect learns 33x faster than RL from scratch and transfers zero-shot to 0.5 mm insertions on a real robot.
What they do
Precise assembly with a multi-fingered hand is contact-rich (hard to teleoperate) and sparse-reward (hard to explore with RL). Play2Perfect splits the problem in two. A goal-conditioned policy first learns to play: in simulation it grasps procedurally generated cuboids and cylinders and moves them through random 6D goal poses. That policy is then finetuned with RL on each assembly step, using only a sparse success reward.
How it works
- Play pretraining. One policy controls a 7-DoF KUKA iiwa and a 22-DoF Sharpa hand. It sees proprioception, the current and goal object pose, and the object’s bounding box. About 1000 random objects with randomized mass and center of mass; each new goal sits near the previous one but with a large rotation, which forces in-hand reorientation instead of moving a fixed grasp.
- Assembly finetuning. Assembly-by-disassembly turns a CAD assembly into a sequence of insertion steps. Each step becomes an RL environment whose only reward is reaching the final part pose.
- Deployment. Zero-shot sim-to-real, with FoundationPose providing object poses.
Results
- Every assembly task is solved in 2-5 h of RL wall-clock time. Scratch RL gets no successes after 24 h, even with dense multi-stage rewards. On a simplified fixtured insertion, dense-reward scratch needs over 100 h to match 4 h of Play2Perfect, a 33x speed-up.
- Ablations: object diversity, full 6D goals (rotation matters most), random goal trajectories and a tight goal threshold all improve finetuning.
- Real robot: insertion succeeds 10/10 at 10 mm clearance, 9/10 at 2 mm and 6/10 at 0.5 mm. Beam assembly steps succeed 16/20 and 12/16; screwing succeeds 5/7 after 7/10 insertions.
Limitations
Short-horizon skills only, with sequencing and goal poses given externally. The policy depends on pose tracking, which breaks under hand occlusion, and sees no vision or touch beyond the object pose.
Videos
Figures
Abstract
Multi-fingered robots promise the speed and dexterity of human hands, yet challenging problems such as precise assembly have remained out of reach. These tasks are contact-rich, making data collection for imitation learning difficult, and sparse-reward, making direct exploration with reinforcement learning (RL) intractable. Consequently, prior work has made progress by structuring the problem with specialized grippers, tool attachments, and environment fixtures. In this work, we argue that before a robot can perfect precise assembly, it must first learn to play. We further ask the question: what factors in the process of learning to play matter for precise assembly? We propose Play2Perfect, an RL framework for task-agnostic pretraining through play on diverse objects and goals, which is then perfected on precise assembly. The goal of play is to acquire reusable manipulation priors, such as grasping, in-hand reorientation and pose reaching. Finetuning then adapts this general prior to assembly, focusing exploration on the final contact-rich, high-precision interactions needed for success. We systematically study key design choices in play pretraining, including object diversity, training objective, trajectory diversity, and goal precision. We show that our prior is 33x more sample-efficient than RL training from scratch, even when provided with dense, multi-stage rewards. We demonstrate zero-shot sim-to-real transfer, achieving 60% success on tight insertions with only 0.5 mm contact clearance, and over 50% success on long-horizon multi-part assembly and screwing.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.