Soft Biomimetic Optical Tactile Sensing with the TacTip: A Review

Nathan F. Lepora

Seminal May 2021

TL;DR A review of the TacTip, a soft optical tactile sensor whose internal pins mimic human skin and lever surface indentation into marker motion seen by an internal camera. It covers a decade of TacTip sensors, hands and skills, with localization accuracy improving from about 1 mm to 0.1-0.5 mm once ConvNets were applied to the tactile images.

What they do

A review of the BRL TacTip as the prototype “SoftBOT” (Soft Biomimetic Optical Tactile) sensor. It relates the design to human fingertip skin, argues that it embodies the shear-sensing hypothesis, and surveys a decade of TacTip sensors, hands and skills, mostly from Bristol Robotics Laboratory.

How it works

Biomimetic skin. A rubber-like outer skin over soft gel, with stiff pins tipped with markers that an internal camera images. The pins mimic the dermal papillae and lever indentation into lateral marker motion.
Signals. Marker displacements stand in for slowly-adapting (Merkel) activity, used for shape and edges; marker velocities stand in for rapidly-adapting (Meissner) activity, used for slip.
Shear sensing. A pin model gives shear strain proportional to pin length times the indentation gradient. This matches the elastomer model behind the shear-sensing hypothesis, but without blurring with depth.
Fabrication. Multi-material 3D printing since 2015 produced variants for OpenHand grippers, the Shadow Modular Grasper, the Pisa/IIT SoftHand, a quadruped foot, whiskers and an event camera (NeuroTac).

Results

Accuracy improved from about 1 mm (probabilistic classifiers on pin displacements) to 0.1-0.5 mm (ConvNets on 128x128 tactile images). 3D surface localization reached 0.1 mm and 0.3 deg.
The Tactile Model O hand classified 26 objects with 93% accuracy and predicted grasp success with 95% accuracy. Slip detection let it re-grasp objects before they dropped.
By averaging pin deflections over repeated measurements, a TacTip detected indentations of 10 microns or less from an ultrasound haptic display.
Deep learning enabled 3D contour following, tactile pushing of unknown objects and braille typing learned with RL.

Limitations

This is a review with no new experiments. Vibration sensing is only partly mimicked with high-frame-rate cameras, and the author questions whether that is effective or biomimetic. Normal and shear strain both move the markers, which complicates inferring contact shape. Deep RL takes many hours of training on a physical robot, so the author expects most learning to move to simulation.

Figures

From Figure 3: Hands from the TacTip family: tactile Model-M2, tactile Model-GR2, tactile Model-O, tactile Shadow Mod... — From Figure 3: Hands from the TacTip family: tactile Model-M2, tactile Model-GR2, tactile Model-O, tactile Shadow Modular Grasper and tactile SoftHand.

From Figure 2: The tactile contact lens and the TacTip work the same way: an array of pins levers a surface indentati... — From Figure 2: The tactile contact lens and the TacTip work the same way: an array of pins levers a surface indentation into lateral (shear) strain.

From Figure 1: TacTip skin pressed onto an edge. The indentation tilts the internal pins, turning it into sideways di... — From Figure 1: TacTip skin pressed onto an edge. The indentation tilts the internal pins, turning it into sideways displacement of the markers on their tips (a shear-strain field).

From Figure 4: A TacTip on a robot arm slides over unknown 3D objects (a head and a lid) under ConvNet-based tactile ... — From Figure 4: A TacTip on a robot arm slides over unknown 3D objects (a head and a lid) under ConvNet-based tactile servo control, with tactile images inset and the traced paths overlaid.

Abstract

Reproducing the capabilities of the human sense of touch in machines is an important step in enabling robot manipulation to have the ease of human dexterity. A combination of robotic technologies will be needed, including soft robotics, biomimetics and the high-resolution sensing offered by optical tactile sensors. This combination is considered here as a SoftBOT (Soft Biomimetic Optical Tactile) sensor. This article reviews the BRL TacTip as a prototypical example of such a sensor. Topics include the relation between artificial skin morphology and the transduction principles of human touch, the nature and benefits of tactile shear sensing, 3D printing for fabrication and integration into robot hands, the application of AI to tactile perception and control, and the recent step-change in capabilities due to deep learning. This review consolidates those advances from the past decade to indicate a path for robots to reach human-like dexterity.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong