← Paper Digest

TaF-Foundation: Wrench-Grounded Cross-Sensor Tactile Representations

Jiaming Jiang, Yuzhe Huang, Baokun Yang, Hao Liang, Li Gao, Haoran Guo, Pei Lin, Shengcheng Luo, Chenxi Xiao, Wanlin Li, Ziyuan Jiao

November 2026

The data-collection rig pressing a probe onto a tactile sensor while a force/torque sensor records the wrench.

TL;DR They record over 100 million tactile readings paired with measured force/torque from 24 different tactile sensors and train one encoder that maps all of them into a shared, force-grounded feature space. Against the strongest baseline it cuts single-frame force error by 17.1% and raises average real-robot success on four tasks from 23.75% to 46.25%.

What they do

Tactile sensors differ in how they work and what they output (gel camera images, magnetic field vectors, pressure maps), so features learned on one rarely transfer to another. TaF-Foundation uses the measured 6D wrench (force and torque) as a shared physical label: each sensor is paired with wrenches on its own, so no sensor-to-sensor pairing is needed. The frozen encoder then feeds a manipulation policy, TaF-Policy.

How it works

  • Data. A Dynamixel-driven rig presses, slides, rotates and rolls more than 200 probe shapes on a sensor while an ATI Axia80 force/torque sensor records the same contact: 101.3M frames and 30,503 episodes from 24 optical, magnetic, pressure and piezoresistive sensors.
  • Encoder. Sensor-specific adapters (residual MLPs on image patches, MLPs for vector signals, a conv front end for pressure matrices) produce tokens plus a sensor-identity token. A shared Transformer initialized from DINOv2 then pools them with a learnable query.
  • Training. A wrench encoder and decoder learn to reconstruct the wrench. The tactile embedding must predict the wrench through that decoder and match an EMA copy of the wrench embedding.
  • TaF-Policy. Frozen features from the left and right tactile sensors join head and wrist camera tokens and robot state in ACT.

Results

  • On 21 seen sensors, single-frame force RMSE is 2.321 N vs 2.799 N (AnyTouch 2), and cross-sensor retrieval R@1 is 34.38% vs 21.27%.
  • On three held-out sensors it has the best overall few-shot score (0.4468 vs 0.4497 for T3), though HTT is better at the 5% and 10% temporal budgets.
  • On a Franka (20 trials per task): 35/60/25/65% on Lift Bottle, Remove Power Bank, Key Insert and Board Clean, vs 20/25/10/40% for AnyTouch 2. Board Clean uses a held-out magnetic sensor. In UniVTAC simulation: 51.33% vs 47.33%.

Limitations

Each new sensor still needs labeled tactile–wrench pairs and a trained input module. Real-world success stays between 25% and 65%: Key Insertion fails on key–keyhole misalignment and Lift Bottle on slippage. Ablations and few-shot results come from a single seed.

Videos

CAD animation of the rig's motions: rotation about x, y and z and translation along z.
Raw contact readings from many of the 24 sensors in the dataset.

Figures

From Figure 1: Over 100 million tactile–wrench pairs from 24 sensors in four sensing families train one shared repres...
From Figure 1: Over 100 million tactile–wrench pairs from 24 sensors in four sensing families train one shared representation, without pairing sensors with each other. It is used for cross-sensor perception, adaptation to unseen sensors and, in TaF-Policy, contact-rich manipulation.
From Figure 4: (a) Sensor-specific adapters and family-specific projections turn each sensor's readings into tokens f...
From Figure 4: (a) Sensor-specific adapters and family-specific projections turn each sensor's readings into tokens for a shared Transformer; a learnable physical query pools them into a 768-dimensional representation. (b) Wrench reconstruction and alignment with EMA wrench embeddings supervise it, and it is evaluated on wrench estimation, cross-sensor retrieval and manipulation with TaF-Policy.
From Figure 5: Few-shot adaptation to three unseen sensors. Curves show temporal wrench-estimation error against the ...
From Figure 5: Few-shot adaptation to three unseen sensors. Curves show temporal wrench-estimation error against the share of adaptation data; the inset gives each method's overall score across adaptation settings.
From Figure 6: The four real-world tasks (bottle lifting, power bank removal, key insertion with Daimon sensors; boar...
From Figure 6: The four real-world tasks (bottle lifting, power bank removal, key insertion with Daimon sensors; board cleaning with a PaXini sensor) and the three UniVTAC simulation tasks (can lifting, key extraction, bottle on shelf).
Abstract

Tactile sensing is essential for contact-rich manipulation, yet heterogeneous sensing mechanisms and observation formats hinder cross-hardware transfer. Since physical contact is governed by Newtonian mechanics, we ground tactile representations in six-dimensional wrenches, establishing a common physical reference across sensing mechanisms. Specifically, we pair tactile observations from all sensors with wrench measurements, enabling shared physical supervision without requiring direct tactile-to-tactile pairing between sensors. To support learning at scale, we develop a dedicated acquisition platform and collect over 100 million paired tactile observations and wrench measurements from 24 sensors spanning four families of sensing principles. Building on this dataset, our framework combines sensor-specific input adaptation with a shared tactile encoder. By learning to reconstruct wrenches, the encoder maps observations from different sensors into a shared, continuous feature space grounded in contact physics. To leverage these representations for manipulation, we introduce TaF-Policy, which integrates tactile features with visual observations and robot states to generate action chunks. We evaluate the learned representations through wrench estimation, cross-sensor retrieval, few-shot adaptation to unseen sensors, and robotic manipulation. On the core benchmark, our method reduces single-frame force RMSE by 17.1% and improves cross-sensor retrieval R@1 by 13.11 percentage points over the strongest baseline. In manipulation experiments, TaF-Policy achieves average success rates of 51.33% across three simulated tasks and 46.25% across four real-world tasks, compared with 47.33% and 23.75%, respectively, for the strongest baselines evaluated on the same task sets. Project page: https://mrjiangjm.github.io/TaF-Foundation/.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.