Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstration
TL;DR Teaches the soft, pneumatic RBO Hand 2 from demonstrations that record only how a human moved the object. RL learns which demonstrations each controller can actually follow, and guided policy search merges the controllers into one neural network policy that turns a valve and pushes abacus beads from several hand positions, close to an oracle given the right demonstrations, and grasps a bottle in 10 of 10 trials.
What they do
The RBO Hand 2 is a cheap, compliant hand driven by inflating air chambers, with no finger position sensing and only on/off valves. Kinesthetic teaching and teleoperation do not work for it. Instead, a human performs the task with their own hand while motion capture records only the object’s motion, and the robot uses reinforcement learning on the real hardware to reproduce those object trajectories.
How it works
- Demonstration selection. Demonstrations and controllers are each modeled as mixtures, and the KL divergence between them is minimized through a variational upper bound. This alternates between softly assigning demonstrations to controllers and optimizing the controllers, so each controller follows the demonstration it can most closely achieve from its starting state and ignores infeasible ones.
- Controller learning. Time-varying linear-Gaussian controllers are optimized with LQG against local linear dynamics fitted to real-robot samples, with a bound on how far each update can move.
- One policy. BADMM-based guided policy search distills the controllers into a single neural network policy.
- System. Valve commands become a continuous signal through 5 Hz pulse-width modulation. The state holds seven chamber pressures, object marker positions and velocities, and sometimes the PR2 arm joint angles.
Results
- Valve: three demonstrations (about 35 degrees of rotation), tested 10 times at each of 4 hand positions. Only this method and the oracle turned the valve consistently at every position; each baseline failed at one or more. The policy switched strategies with position: last two fingers, thumb, or index finger.
- Abacus: tested 10 times at each of 3 positions. It moved the target bead closer to the demonstrated motion than the single-demonstration and hand-designed baselines, matched only by the oracle.
- Bottle grasp: 10/10, the same as a hand-designed controller.
Limitations
Object state comes from LED-marker motion capture; the authors name visual tracking and policies using only pressure or tactile sensing as future work. In the grasping task the arm motion is scripted and only the fingers are learned.
Videos
Figures
Abstract
Dexterous multi-fingered hands can accomplish fine manipulation behaviors that are infeasible with simple robotic grippers. However, sophisticated multi-fingered hands are often expensive and fragile. Low-cost soft hands offer an appealing alternative to more conventional devices, but present considerable challenges in sensing and actuation, making them difficult to apply to more complex manipulation tasks. In this paper, we describe an approach to learning from demonstration that can be used to train soft robotic hands to perform dexterous manipulation tasks. Our method uses object-centric demonstrations, where a human demonstrates the desired motion of manipulated objects with their own hands, and the robot autonomously learns to imitate these demonstrations using reinforcement learning. We propose a novel algorithm that allows us to blend and select a subset of the most feasible demonstrations to learn to imitate on the hardware, which we use with an extension of the guided policy search framework to use multiple demonstrations to learn generalizable neural network policies. We demonstrate our approach on the RBO Hand 2, with learned motor skills for turning a valve, manipulating an abacus, and grasping.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.