DenseTact: Optical Tactile Sensor for Dense Shape Reconstruction
TL;DR A hemispherical camera-based fingertip that reconstructs the 3D shape of its whole contact surface from one fisheye image using a DenseNet encoder-decoder. Mean depth error on unseen shapes is 0.28 mm at 18 ms per image, for under $80 per sensor.
What they do
High-resolution camera-based tactile sensors such as GelSight, GelSlim and DIGIT are flat, which limits in-hand manipulation, and curved alternatives are costly or low-resolution. DenseTact is a hemispherical fingertip: a fisheye camera looks at the inside of a soft, reflective-coated silicone dome, and a neural network turns each image into a depth map of the entire 3D surface.
How it works
- Hardware. A 25 mm-radius, 35 mm-tall dome of extra-soft silicone with a metallic-ink coating, an 8 MP camera with a 185° fisheye lens, and three lit RGB LEDs whose colors encode surface shape. Parts cost under $80, $70 of it the camera.
- Ground truth. Instead of measuring the surface with a range sensor, a CNC machine presses 3D-printed shells and indenters into the dome so the deformed shape is known everywhere. A Gaussian-process fit maps image radius onto the sphere, and ray casting against the printed STL gives each of 253,213 pixels its radial depth.
- Model. An encoder-decoder with skip connections and a pretrained DenseNet-161 encoder predicts the depth image from the RGB image. It is trained with L1, depth-gradient and SSIM losses on 29,200 images from 40 hours of data collection.
Results
- On 1,000 test images with shell and indenter combinations not used in training, mean L1 depth error is 0.28 mm (0.24 mm on training images). The 3D printer’s own error is about 0.11 mm.
- Inference takes 18 ms per image, enough for the camera’s 30 fps.
- Two sensors on an Allegro hand pinch a known sphere: over 23 grasps, ICP between the sensor point clouds and the object model gives an average fitness (inlier ratio) of 0.597.
- No noticeable change in the sensor after 30,000 presses.
Limitations
Ground truth is only as accurate as the 3D printer, and indenters were rotated by hand on two axes during data collection, adding error. The authors call for a larger, more varied dataset to reduce bias. The sensor reconstructs shape only; contact-force estimation is left to future work.
Videos
Figures
Abstract
Increasing the performance of tactile sensing in robots enables versatile, in-hand manipulation. Vision-based tactile sensors have been widely used as rich tactile feedback has been shown to be correlated with increased performance in manipulation tasks. Existing tactile sensor solutions with high resolution have limitations that include low accuracy, expensive components, or lack of scalability. In this paper, an inexpensive, scalable, and compact tactile sensor with high-resolution surface deformation modeling for surface reconstruction of the 3D sensor surface is proposed. By measuring the image from the fisheye camera, it is shown that the sensor can successfully estimate the surface deformation in real-time (1.8ms) by using deep convolutional neural networks. This sensor in its design and sensing abilities represents a significant step toward better object in-hand localization, classification, and surface estimation all enabled by high-resolution shape reconstruction.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.