Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks

Michelle A. Lee, Yuke Zhu, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg, Jeannette Bohg

★ Must read October 2018

TL;DR A self-supervised network fuses camera images, force-torque readings and proprioception into a compact representation, and a peg-insertion policy is learned on top of it with RL on a real robot. In simulation the full model completes nearly 80% of insertions, while every baseline missing vision or touch completes under 5%.

What they do

Peg insertion needs vision to find the hole and touch to align and insert, but deep RL on raw images and forces needs too many samples for a real robot. The authors first learn a compact representation of vision, force and proprioception with self-supervision, then freeze it and train a small policy on top with RL.

How it works

Encoders. A 6-layer CNN encodes 128x128 RGB images from a fixed camera, causal convolutions encode the last 32 readings of a wrist force-torque sensor, and an MLP encodes end-effector position and velocity. A fusion MLP outputs a 128-d representation.
Self-supervised objectives. Given the next action, predict the optical flow it will cause and whether the end-effector will make contact in the next control cycle; also classify whether the visual and haptic streams are time-aligned. Labels come from robot kinematics and simple force heuristics; the training set is 100k states, collected in 90-120 minutes.
Policy and control. TRPO trains a 2-layer MLP that outputs 3D end-effector displacements at 20 Hz, with a staged reward; freezing the representation leaves 3% of the parameters to learn. An impedance and operational-space controller turns the displacements into compliant torque commands at 200 Hz on a KUKA iiwa.

Results

Simulation (square peg): the full model completes nearly 80% of insertions; without vision, without haptics, or with proprioception only, completion stays under 5%. The no-haptics model gets the peg into the hole about half the time but rarely finishes.
Real robot: each policy trains for 300 episodes (about 5 hours) on round, triangular or semicircular pegs with about 2 mm clearance, and succeeds at a level similar to simulation.
Transfer: the triangular-peg policy succeeds over 60% of the time on unseen hexagonal and square pegs. Training a new policy on the frozen triangular-peg representation raises this by 19% and 30%.
The policy recovers from camera occlusion and pushes to the arm (shown qualitatively).

Limitations

Actions are 3D position displacements; full 6-DoF control of position and orientation is left to future work. Experiments cover only peg insertion, use a hand-designed staged reward, and train a separate representation and policy for each peg shape.

Videos

Figures

From Figure 1: Vertical force readings and camera views over the stages of a peg insertion: free-space motion, first ... — From Figure 1: Vertical force readings and camera views over the stages of a peg insertion: free-space motion, first contact with the box, sliding on the surface while aligning, a force peak at the hole edge, and a drop as the peg slides in.

From Figure 2: The representation network encodes RGB images, force-torque readings over a 32 ms window, and end-effe... — From Figure 2: The representation network encodes RGB images, force-torque readings over a 32 ms window, and end-effector position and velocity, then fuses them into one multimodal representation trained end-to-end with self-supervision.

From Figure 4: RL training curves for simulated peg insertion with the full model and with representations trained wi... — From Figure 4: RL training curves for simulated peg insertion with the full model and with representations trained without vision, without haptics, or without either.

From Figure 6: Real-robot peg insertion outcomes per peg shape; the last four bars transfer policies and representati... — From Figure 6: Real-robot peg insertion outcomes per peg shape; the last four bars transfer policies and representations from trained pegs to novel peg shapes.

Abstract

Contact-rich manipulation tasks in unstructured environments often require both haptic and visual feedback. However, it is non-trivial to manually design a robot controller that combines modalities with very different characteristics. While deep reinforcement learning has shown success in learning control policies for high-dimensional inputs, these algorithms are generally intractable to deploy on real robots due to sample complexity. We use self-supervision to learn a compact and multimodal representation of our sensory inputs, which can then be used to improve the sample efficiency of our policy learning. We evaluate our method on a peg insertion task, generalizing over different geometry, configurations, and clearances, while being robust to external perturbations. Results for simulated and real robot experiments are presented.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong