← Paper Digest

CoRA-VTLA: Contact-Guided Attention and Queryable Response Memory for Tactile Post-Training

Chengye Song, Rui Song, Shengqi Wang, Jiawei Zhang, Hongzhou Wang, Xueyang Guo, Yang Liu

November 2026

Real robot inserting a test tube into a rack.

TL;DR CoRA-VTLA post-trains a π0.5 VLA with fingertip pressure and wrist force, adding a memory of how contact responded to recent actions and a loss that steers attention to the contact region. It reaches 66.7% mean success on three real-robot precision tasks, against 44.7% for the best baseline.

What they do

Cameras cannot tell free motion from a jammed part once contact begins, and tactile data is scarce in VLA pretraining. CoRA-VTLA adds lightweight tactile modules to π0.5 and post-trains it on demonstrations that include fingertip pressure arrays and wrist force/torque, for precision tasks such as insertion, stamping and stacking soft cups.

How it works

  • Contact token. An encoder fuses both fingertip pressure maps and the wrist wrench into one token for the action expert.
  • Action-response memory. After each action it stores the change in pressure and wrench, plus the wrench relative to the episode start. A temporal Transformer encodes the last 16 events (0.8 s in MuJoCo), and a query built from current contact and robot state reads them out, so the policy can tell first contact from a sustained jam.
  • Contact-guided gaze. A tactile semantic token (contact, stability, load change) enters the VLM prefix, and a loss pushes its attention toward the image patches where active fingertip cells project.
  • Training. The contact and memory encoders are first pretrained to predict the response to a demonstrated command, then the whole model is fine-tuned with action, semantic and gaze losses.

Results

  • MuJoCo nail, plug and knob tasks (150 trials): 73.33% success, against 31.33% for π0.5, 50% for direct tactile fusion and 44% for adapted FTP-π0.5. Plug insertion rises from 0% to 76% over π0.5.
  • UniVTAC Insert Tube with optical tactile images: 67%, against 46% (π0.5) and 51% (direct fusion).
  • UR5e with fingertip e-skin, 50 trials per task: 68% cup stacking, 76% stamping, 56% tube insertion; mean 66.7% against 44.7% for the best baseline.
  • Ablations: without the memory token or response pretraining, MuJoCo success drops to 54%; without gaze supervision, to 49.33%.

Limitations

Recovery is not always safe: four safety failures remain on nail insertion, and the authors name safer recovery and a longer memory horizon as future work. Adapted FTP-π0.5 still does better on knob operation (72% vs 66%). The simulation tasks are custom, with one task from a public benchmark.

Videos

Real robot inking a stamp and pressing it onto paper.

Figures

From Figure 2: CoRA-VTLA architecture and training. (a) A semantic token joins the VLM prefix; the contact and memory...
From Figure 2: CoRA-VTLA architecture and training. (a) A semantic token joins the VLM prefix; the contact and memory tokens join the action-expert suffix. (b) Response prediction initializes the contact and memory encoders, followed by joint post-training with action, semantic and gaze supervision.
From Figure 3: Real-robot platform (workspace, arm and fingertip pads) and keyframes of CoRA-VTLA stacking soft paper...
From Figure 3: Real-robot platform (workspace, arm and fingertip pads) and keyframes of CoRA-VTLA stacking soft paper cups, stamping on paper and inserting a test tube. The highlighted frames show the tube catching on the rim before the robot backs off and realigns.
From Figure 4: Plug recovery with and without the memory token in a matched MuJoCo scene: withdrawal and the second i...
From Figure 4: Plug recovery with and without the memory token in a matched MuJoCo scene: withdrawal and the second insertion attempt, with curves of minimum pin depth and change in wrist force.
Abstract

In precision manipulation, small errors in alignment or force can cause jamming or deformation, making contact feedback essential for adapting pretrained vision-language-action (VLA) models to these tasks. A single tactile observation leaves the response to recent actions ambiguous. Direct tactile fusion during post-training can also disperse the action expert's visual attention and reduce the share it allocates to contact regions. To address these challenges, CoRA-VTLA augments pretrained VLAs with lightweight tactile modules. A queryable action-response memory retrieves recent contact responses so that the action expert can combine them with immediate tactile feedback. Contact-guided gaze supervision concentrates the visual attention of the tactile semantic token on image regions corresponding to fingertip contact. CoRA-VTLA achieves 73.33% mean success across three MuJoCo tasks, compared with 50% for direct tactile fusion with the same sensor inputs. It also achieves 67% success on an optical-tactile task from the public UniVTAC benchmark and 66.7% mean success across three real-robot tasks. These results show that tactile post-training helps pretrained VLAs adjust to changing contact.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.