← Paper Digest

TapTacTouch: Shape Completion from Sparse Tactile Contacts with Geometric Priors

Yoobin Shin, Jaeyoung Lee, Jieun Kang, Hyun Song, Woojin Kim, Andrew Jaeyong Choi

November 2026

TL;DR A robot arm with a 20-DoF hand probes an object with 3x3 pressure pads on its fingers and fits a multi-scale Gaussian process implicit surface to the contacts, with no camera and no training. On six 3D-printed objects, with contacts projected onto the true surface for evaluation, the meshes reach 1.61 mm Chamfer distance and a 95% F-Score.

What they do

TapTacTouch reconstructs a 3D object mesh from touch alone. A 6-DoF ROBOTIS OMY arm carries a 20-DoF HX5 hand with a 3x3 pressure array on each finger pad. Active taxels are mapped to 3D contact points through forward kinematics and fed to a Gaussian Process Implicit Surface (GPIS). No camera, learned shape model or object class is used.

How it works

  • Scripted exploration. A rule-based routine finds the object’s lower and upper boundary, presses the top on a 7x4 grid, then sweeps the sides layer by layer at 10 mm intervals over 18 wrist angles, using the four non-thumb fingers.
  • Multi-scale GPIS. Contacts and their normals become signed-distance constraints. A fine and a coarse RBF kernel are mixed, so contacts keep local detail while distant ones still inform each other.
  • Reliability and pseudo-contacts. Side contacts are scored for consistency with their neighbours; doubtful ones get larger noise. Midpoints between nearby reliable contacts with similar normals are added as weak surface points.
  • Cross-section prior. When the scanned layers have consistent profiles, their convex hulls are merged into an expanded polygonal prism that serves as the GP mean in unobserved regions. The GP residual then fits it to the contacts, and Marching Cubes extracts the mesh.

Results

  • On six 3D-printed objects (8-10 cm wide, up to 6 cm tall): mean Chamfer distance 1.610 mm, EMD 7.738 mm and F-Score 95.00% at 5 mm. The worst object reaches 2.602 mm and 84.49%.
  • Reconstruction takes 49.99 s on average once contacts are collected; exploration takes about 30 min per object.
  • In unobserved regions, the full method lowers Chamfer distance from 2.133 to 1.992 mm and raises F-Score from 89.705% to 92.605% over plain multi-scale GPIS. Single-scale GPIS gets 4.076 mm and 71.253%.

Limitations

The reported metrics use ground-truth-assisted inputs: contacts are projected onto the true surface and perturbed with 0.5 mm noise, so they do not reflect raw sensing error. Exploration follows a fixed, slow sequence, the object must stay still on a mount, and only six flat-bottomed objects in a narrow size range are tested.

Figures

From Figure 1: The tactile-only setup. Contact points from the hand form a 3D point cloud that a Gaussian Process Imp...
From Figure 1: The tactile-only setup. Contact points from the hand form a 3D point cloud that a Gaussian Process Implicit Surface (GPIS) turns into a mesh; below, ground-truth 3D prints and their reconstructions.
From Figure 2: (a) The arm and hand collect tactile contacts and joint states, mapped to world-frame contacts and nor...
From Figure 2: (a) The arm and hand collect tactile contacts and joint states, mapped to world-frame contacts and normals by forward kinematics. (b) Pseudo-Contact Generation adds points, the Cross-Section Prior sets the GP mean, multi-scale GP regression fits a residual field, and Marching Cubes extracts the mesh.
From Figure 3: The hardware: a ROBOTIS OMY arm, a ROBOTIS HX5-D20-MRT hand, and the tactile sensors on the finger pads.
From Figure 3: The hardware: a ROBOTIS OMY arm, a ROBOTIS HX5-D20-MRT hand, and the tactile sensors on the finger pads.
From Figure 4: Input contacts, reconstructed meshes and ground-truth meshes for Objects 1-6, each shown from the same...
From Figure 4: Input contacts, reconstructed meshes and ground-truth meshes for Objects 1-6, each shown from the same viewpoint and scale.
Abstract

3D object reconstruction is essential for robotic perception and manipulation, yet conventional approaches predominantly rely on visual observations that can be limited by occlusion or unavailable in visually constrained environments. In this work, we propose a vision-free, training-free, and category-agnostic framework for reconstructing 3D object meshes from sparse tactile contacts without relying on pretrained shape models. A 6-DoF robotic arm equipped with a 20-DoF multi-finger robotic hand collects tactile observations using 3x3 pressure sensor arrays mounted on the finger pads. The acquired tactile measurements are transformed into 3D contact constraints through robot kinematics and integrated into a multi-scale Gaussian Process Implicit Surface (GPIS). To address sparse and incomplete surface observations, the proposed method incorporates contact confidence weighting, weak pseudo-surface constraints, and an observation-driven geometric prior to preserve local surface geometry and provide structural guidance in unobserved regions. Real-world experiments on six objects achieve a Chamfer Distance (CD) of 1.610, an Earth Mover's Distance (EMD) of 7.738, and an F-Score of 95.00, with reconstruction completed in an average of 49.99 s per object. These results demonstrate that sparse tactile sensing can provide sufficient geometric information for continuous 3D object reconstruction without visual or semantic information.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.