TensorTouch: Calibration of Tactile Sensors for High Resolution Stress Tensor and Deformation for Dexterous Manipulation

Check out our website: TensorTouch! Arxiv here

A diagram of our method.

Abstract:

TensorTouch is a comprehensive framework for stress tensor estimation from 3D optical tactile sensors designed specifically for complex manipulation tasks. The framework is developed using a physics-based calibration methodology that combines motion capture data collection, finite element analysis, and deep learning to extract detailed contact information from tactile images. From a single undeformed/deformed image of the sensor, TensorTouch estimates dense stress tensors, contact shear and normal force, and displacement distribution pixelwise. The TensorTouch framework achieves sub-millimeter position accuracy and precise force estimation while supporting large sensor deformations crucial for manipulating soft objects. Experimental validation demonstrates 90% success in selectively grasping one of two strings based on detected motion, showcasing the framework’s capabilities for dexterous manipulation tasks.