OmniTact: A Multi-Directional High Resolution Touch Sensor
TL;DR OmniTact casts a GelSight-style gel skin directly over five micro-cameras to make a thumb-sized fingertip with high-resolution touch on all sides. With both its top and side cameras, a learned policy inserts an electrical connector in 80% of trials, against 17% with an OptoForce sensor.
What they do
Optical tactile sensors like GelSight give high-resolution images but only on one flat face, while curved multi-directional sensors (capacitive, resistive, photodiode arrays, BioTac) have far lower resolution. OmniTact is a thumb-shaped fingertip that gets both: several micro-cameras look at a curved gel skin from the inside. The authors test it on contact-angle estimation and on inserting an electrical connector using touch alone.
How it works
- Cameras. Five endoscope cameras (1.35 x 1.35 x 5 mm, about 90° field of view, 5 mm minimum focus, 400x400 pixels at 30 fps) sit in a 3D-printed mount: one facing up, four facing sideways. Together they cover 270° vertically and 360° horizontally, apart from small blind spots. The sensor is 30 mm in diameter with a 33 mm tall sensitive area.
- Gel. Unlike GelSight, the silicone is cast directly around the cameras with no support plate, then coated with aluminum powder mixed into the same silicone. Red, green and blue LEDs next to each camera, on a flexible PCB, light the gel from different directions.
- Learning. No explicit normal or geometry estimation. Each camera image goes through its own ImageNet-pretrained ResNet; the features are concatenated and fully connected layers regress the contact angle or the end-effector target, trained with an MSE loss.
Results
- Contact angle (1000 samples per range, median absolute error): OmniTact gets 1.14°, 1.99° and 1.25° for 0-22.5°, 22.5-60° and 60-90°. A flat GelSight is better at shallow angles (0.33°) but worse at steeper ones (4.23° and 1.99°).
- Connector insertion (100 keyboard demonstrations, 30 trials each): top and side cameras together succeed 80% of the time, top only 67%, side only 50%, OptoForce 17%.
- From the failures, the authors conclude the top camera localizes the hand relative to the outlet via the floor texture, and the side camera localizes the plug in the gripper.
Limitations
Cost: each camera is US$600, US$3200 for the five-camera prototype. The design has small blind spots, and all LEDs stay on, so light from neighboring cameras’ LEDs overlaps. Experiments use only two of the five cameras (top and one side). In the insertion task the plug starts at one of five positions spanning 5.5 mm, plus a random gripper offset drawn from an 8 mm range in x and in y.
Figures
Abstract
Incorporating touch as a sensing modality for robots can enable finer and more robust manipulation skills. Existing tactile sensors are either flat, have small sensitive fields or only provide low-resolution signals. In this paper, we introduce OmniTact, a multi-directional high-resolution tactile sensor. OmniTact is designed to be used as a fingertip for robotic manipulation with robotic hands, and uses multiple micro-cameras to detect multi-directional deformations of a gel-based skin. This provides a rich signal from which a variety of different contact state variables can be inferred using modern image processing and computer vision methods. We evaluate the capabilities of OmniTact on a challenging robotic control task that requires inserting an electrical connector into an outlet, as well as a state estimation problem that is representative of those typically encountered in dexterous robotic manipulation, where the goal is to infer the angle of contact of a curved finger pressing against an object. Both tasks are performed using only touch sensing and deep convolutional neural networks to process images from the sensor's cameras. We compare with a state-of-the-art tactile sensor that is only sensitive on one side, as well as a state-of-the-art multi-directional tactile sensor, and find that OmniTact's combination of high-resolution and multi-directional sensing is crucial for reliably inserting the electrical connector and allows for higher accuracy in the state estimation task. Videos and supplementary material can be found at https://sites.google.com/berkeley.edu/omnitact
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.