Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation

Alex Church, John Lloyd, Raia Hadsell, Nathan F. Lepora

Seminal June 2021

TL;DR Train tactile RL policies in PyBullet on simulated depth images of a TacTip sensor, then run them on a real robot by translating real tactile images into simulated ones with a pix2pix GAN. Zero-shot transfer works on 4 of 5 tasks, including 100 consecutive successful ball-rolling trials.

What they do

An open-source suite of PyBullet RL environments for an optical tactile sensor (the TacTip), with five tasks: edge following, surface following, object rolling, object pushing and object balancing. Policies are learned with PPO from simulated tactile images and transferred zero-shot to a UR5 by translating real sensor images into simulated ones.

How it works

Fast tactile simulation. The soft tip is approximated as a rigid body with soft contact. A synthetic camera inside the tip’s CAD model renders depth, and subtracting a no-contact reference gives a penetration map; up to 1000 fps at 128×128 on a 2080Ti. Only contact geometry is simulated, not the real marker images or shear.
RL. Stable-Baselines-3 PPO with a CNN on tactile images, compared against simulator-state, visual and visual+tactile observations over 3 seeds.
Real-to-sim translation. Per task, the same static data collection runs in sim and reality: 5000 training pairs, about 5 h on hardware versus about 100 s in sim. The real sensor is slid randomly during collection so the translation ignores shear. A pix2pix GAN (spectral instead of instance normalisation) maps real images to sim images, which the policy then acts on.

Results

Tactile observations trained more sample-efficiently and stably than simulator state on edge following and rolling; vision alone performed worst.
Generated images match their simulated targets with mean SSIM above 0.99 on all three validation sets. On supervised edge angle and position prediction, MAE is about 3x and nearly 4x lower than the earlier elastic-deformation TacTip simulation.
Edge following on four unseen shapes stays 1.09-1.58 mm from the true edge in reality (0.47-1.38 mm in sim); surface following has 0.57 mm depth error (0.30 mm in sim).
Rolling 2-8 mm ball bearings: 100 consecutive successful real trials, to within about 2 mm. Pushing succeeds on several trajectories with new object shapes, at 9.9-16.7 mm mean distance from the path.

Limitations

Balancing did not transfer: the simulated policy needed physics parameters beyond the UR5. Real edge following oscillates more, which the authors attribute to latency and PPO action clipping. Pushing was sensitive to initial conditions and perturbations. Each task needs its own GAN dataset, and shear, texture and soft-body deformation are not simulated.

Figures

From Figure 1: (a) a policy is learned in simulation from simulated tactile images; (b) a GAN is trained on paired re... — From Figure 1: (a) a policy is learned in simulation from simulated tactile images; (b) a GAN is trained on paired real and simulated images of a static environment; (c) on the real robot, real images pass through the generator and then the policy.

From Figure 2: the five environments (edge following, surface following, ball rolling, cube pushing, object balancing... — From Figure 2: the five environments (edge following, surface following, ball rolling, cube pushing, object balancing) and PPO training reward for each observation type, averaged over 3 seeds.

From Figure 3: sim and real edge-following trajectories around four flat shapes (square, circle, clover, foil) used t... — From Figure 3: sim and real edge-following trajectories around four flat shapes (square, circle, clover, foil) used to test generalisation.

From Figure 5: real and simulated trajectories rolling ball bearings of 2-8 mm diameter from a start position to a goal.

Abstract

Simulation has recently become key for deep reinforcement learning to safely and efficiently acquire general and complex control policies from visual and proprioceptive inputs. Tactile information is not usually considered despite its direct relation to environment interaction. In this work, we present a suite of simulated environments tailored towards tactile robotics and reinforcement learning. A simple and fast method of simulating optical tactile sensors is provided, where high-resolution contact geometry is represented as depth images. Proximal Policy Optimisation (PPO) is used to learn successful policies across all considered tasks. A data-driven approach enables translation of the current state of a real tactile sensor to corresponding simulated depth images. This policy is implemented within a real-time control loop on a physical robot to demonstrate zero-shot sim-to-real policy transfer on several physically-interactive tasks requiring a sense of touch.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong