← Paper Digest

TacInno: Teaching Vision-Language-Action Models the Contact Dynamics Beyond Vision and Action

Weile Li, Yunfan Lou, Muqing Shi, Zihao Zhang, Zhenbiao Huang, Boxian Yao, Xiyu Zhu, Zhibo Pang

November 2026

Real Franka FR3 with a fingertip tactile sensor wiping a board.

TL;DR TacInno trains a tactile VLA to predict only the part of the future touch signal that the images and the robot's own actions cannot explain, and uses it purely as a training target. It reaches 87.5% average success on six simulated tasks and 70.5% on four real Franka tasks, 10.7 and 9.0 points above the strongest visuo-tactile baseline.

What they do

Visuo-tactile VLAs often gain little from touch. Much of what a gel sensor sees, such as the gel deforming as the gripper closes, already follows from the camera images and the robot’s own actions. TacInno isolates the remainder, which the authors call tactile innovation: the tactile change left after conditioning on vision and executed actions. A policy built on π0.5 learns to predict it, as a training target only.

How it works

  • Tokenizer. A VQ-VAE-style encoder-decoder (spatial-temporal Transformers on frozen DINOv2 image features) compresses tactile frames at t, t+5, t+10 and t+15 into a 16-bit code (4 tokens of 16 values each). Both halves see the images and executed actions, so the code only has to carry what those cannot explain.
  • Policy. TacInno-VLA adds 60 tactile tokens (from tactile frames at t-8, t-4 and t) and four code queries to π0.5’s action expert. The queries predict the code. An attention mask stops action tokens from reading them, so the loss shapes the shared tactile features. A second head predicts contact state (area, shear, depth).
  • Deployment. The tokenizer and both heads are dropped, leaving π0.5 plus the tactile branch.

Results

  • UniVTAC simulation (6 tasks, 50 demos each): 87.5% average vs 76.8% for SimpleTouch and 75.3% for N0-VTLA. It is the only policy at 75% or above on every task.
  • Tactile-off test: swapping touch for a no-contact frame drops TacInno-VLA from 88% to 0% on Lift Can, while the baselines keep most of their success.
  • Ablations on the lifting tasks: removing vision and action conditioning from the tokenizer costs 21 points; letting actions attend to the code queries costs 15.
  • Real Franka FR3 with a Xense fingertip sensor (4 tasks, 50 trials each): 70.5% vs 61.5% for SimpleTouch and 58.5% for FTP-1. Without the code loss, 47.5%.

Limitations

The paper has no limitations section. Real-world plug insertion reaches only 38%, and it ties FTP-1 at 88% on board wiping. In simulation it trails a baseline on Lift Bottle (78 vs 82) and Put Bottle (75 vs 88). Training adds a separate stage: a tokenizer must first label every demonstration frame.

Videos

Simulated Lift Can, with the two tactile images shown on the right.
Tactile-off test on Lift Can: with touch (left) and with a fixed no-contact tactile frame (right).

Figures

From Figure 1: Overview of TacInno. (a) The tactile change mixes a part explained by vision and executed actions with...
From Figure 1: Overview of TacInno. (a) The tactile change mixes a part explained by vision and executed actions with a contact response that only touch reveals. (b) The tokenizer encodes that response into the tactile innovation code. (c) Existing visuo-tactile VLAs versus TacInno, which uses the code only as a training target. (d) Average and lowest per-task success on six UniVTAC tasks. (e) Success on four real-world tasks.
From Figure 2: The TacInno tokenizer. The encoder compresses the tactile observations from t to t+15 into a 16-bit co...
From Figure 2: The TacInno tokenizer. The encoder compresses the tactile observations from t to t+15 into a 16-bit code, and the decoder reconstructs the future tactile change from the current tactile frame and the code. Both are conditioned on visual observations and executed actions, so the code keeps only tactile innovation.
From Figure 3: TacInno-VLA. Tactile tokens and four learnable code queries are inserted into the action expert of pi0...
From Figure 3: TacInno-VLA. Tactile tokens and four learnable code queries are inserted into the action expert of pi0.5. During training, two auxiliary heads predict the innovation code from the code queries and the current contact state from the tactile tokens. The attention mask (left) stops all other tokens from attending to the code queries. At deployment (top left), the tokenizer and both heads are removed.
From Figure 4: Tactile innovation over a held-out Lift Can demonstration. Left: at five time steps, the head-camera i...
From Figure 4: Tactile innovation over a held-out Lift Can demonstration. Left: at five time steps, the head-camera image, the tactile change over the next 15 steps, and the parts explained by vision and executed actions (blue) and by the code (orange). Right: the size of the tactile change (gray), the fraction explained by the code (orange) and the can tilt (dashed). The code's share peaks before contact onset (a) and at incipient slip (c).
Abstract

Vision-Language-Action (VLA) models have recently been equipped with tactile sensing to perceive visually subtle contact events in contact-rich manipulation. However, adding tactile input yields inconsistent gains, and some visuo-tactile VLA models rely little on tactile input, because much of the tactile change, such as the gel deformation during gripper closure, is predictable from visual observations and executed actions. To address this problem, we introduce TacInno, which isolates the contact dynamics that vision and executed actions cannot explain, termed tactile innovation, and uses tactile innovation only as a training target. Specifically, the TacInno tokenizer, conditioned on visual observations and executed actions, compresses the future tactile change into a discrete code, and the policy, TacInno-VLA, learns to predict the code, adding no module for the code at deployment. TacInno-VLA achieves average success rates of 87.5% across six simulation tasks and 70.5% across four real-world tasks, outperforming the strongest visuo-tactile baseline by 10.7 and 9.0 percentage points, respectively. Ablation studies validate the contributions of visual and action conditioning, multi-frame encoding, and auxiliary code prediction. Together, these results demonstrate the effectiveness of tactile innovation as a supervision signal for contact-rich manipulation.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.