TacInno: Teaching Vision-Language-Action Models the Contact Dynamics Beyond Vision and Action
TL;DR TacInno trains a tactile VLA to predict only the part of the future touch signal that the images and the robot's own actions cannot explain, and uses it purely as a training target. It reaches 87.5% average success on six simulated tasks and 70.5% on four real Franka tasks, 10.7 and 9.0 points above the strongest visuo-tactile baseline.
What they do
Visuo-tactile VLAs often gain little from touch. Much of what a gel sensor sees, such as the gel deforming as the gripper closes, already follows from the camera images and the robot’s own actions. TacInno isolates the remainder, which the authors call tactile innovation: the tactile change left after conditioning on vision and executed actions. A policy built on π0.5 learns to predict it, as a training target only.
How it works
- Tokenizer. A VQ-VAE-style encoder-decoder (spatial-temporal Transformers on frozen DINOv2 image features) compresses tactile frames at t, t+5, t+10 and t+15 into a 16-bit code (4 tokens of 16 values each). Both halves see the images and executed actions, so the code only has to carry what those cannot explain.
- Policy. TacInno-VLA adds 60 tactile tokens (from tactile frames at t-8, t-4 and t) and four code queries to π0.5’s action expert. The queries predict the code. An attention mask stops action tokens from reading them, so the loss shapes the shared tactile features. A second head predicts contact state (area, shear, depth).
- Deployment. The tokenizer and both heads are dropped, leaving π0.5 plus the tactile branch.
Results
- UniVTAC simulation (6 tasks, 50 demos each): 87.5% average vs 76.8% for SimpleTouch and 75.3% for N0-VTLA. It is the only policy at 75% or above on every task.
- Tactile-off test: swapping touch for a no-contact frame drops TacInno-VLA from 88% to 0% on Lift Can, while the baselines keep most of their success.
- Ablations on the lifting tasks: removing vision and action conditioning from the tokenizer costs 21 points; letting actions attend to the code queries costs 15.
- Real Franka FR3 with a Xense fingertip sensor (4 tasks, 50 trials each): 70.5% vs 61.5% for SimpleTouch and 58.5% for FTP-1. Without the code loss, 47.5%.
Limitations
The paper has no limitations section. Real-world plug insertion reaches only 38%, and it ties FTP-1 at 88% on board wiping. In simulation it trails a baseline on Lift Bottle (78 vs 82) and Put Bottle (75 vs 88). Training adds a separate stage: a tokenizer must first label every demonstration frame.
Videos
Figures




Abstract
Vision-Language-Action (VLA) models have recently been equipped with tactile sensing to perceive visually subtle contact events in contact-rich manipulation. However, adding tactile input yields inconsistent gains, and some visuo-tactile VLA models rely little on tactile input, because much of the tactile change, such as the gel deformation during gripper closure, is predictable from visual observations and executed actions. To address this problem, we introduce TacInno, which isolates the contact dynamics that vision and executed actions cannot explain, termed tactile innovation, and uses tactile innovation only as a training target. Specifically, the TacInno tokenizer, conditioned on visual observations and executed actions, compresses the future tactile change into a discrete code, and the policy, TacInno-VLA, learns to predict the code, adding no module for the code at deployment. TacInno-VLA achieves average success rates of 87.5% across six simulation tasks and 70.5% across four real-world tasks, outperforming the strongest visuo-tactile baseline by 10.7 and 9.0 percentage points, respectively. Ablation studies validate the contributions of visual and action conditioning, multi-frame encoding, and auxiliary code prediction. Together, these results demonstrate the effectiveness of tactile innovation as a supervision signal for contact-rich manipulation.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
