CoRA-VTLA: Contact-Guided Attention and Queryable Response Memory for Tactile Post-Training
TL;DR CoRA-VTLA post-trains a π0.5 VLA with fingertip pressure and wrist force, adding a memory of how contact responded to recent actions and a loss that steers attention to the contact region. It reaches 66.7% mean success on three real-robot precision tasks, against 44.7% for the best baseline.
What they do
Cameras cannot tell free motion from a jammed part once contact begins, and tactile data is scarce in VLA pretraining. CoRA-VTLA adds lightweight tactile modules to π0.5 and post-trains it on demonstrations that include fingertip pressure arrays and wrist force/torque, for precision tasks such as insertion, stamping and stacking soft cups.
How it works
- Contact token. An encoder fuses both fingertip pressure maps and the wrist wrench into one token for the action expert.
- Action-response memory. After each action it stores the change in pressure and wrench, plus the wrench relative to the episode start. A temporal Transformer encodes the last 16 events (0.8 s in MuJoCo), and a query built from current contact and robot state reads them out, so the policy can tell first contact from a sustained jam.
- Contact-guided gaze. A tactile semantic token (contact, stability, load change) enters the VLM prefix, and a loss pushes its attention toward the image patches where active fingertip cells project.
- Training. The contact and memory encoders are first pretrained to predict the response to a demonstrated command, then the whole model is fine-tuned with action, semantic and gaze losses.
Results
- MuJoCo nail, plug and knob tasks (150 trials): 73.33% success, against 31.33% for π0.5, 50% for direct tactile fusion and 44% for adapted FTP-π0.5. Plug insertion rises from 0% to 76% over π0.5.
- UniVTAC Insert Tube with optical tactile images: 67%, against 46% (π0.5) and 51% (direct fusion).
- UR5e with fingertip e-skin, 50 trials per task: 68% cup stacking, 76% stamping, 56% tube insertion; mean 66.7% against 44.7% for the best baseline.
- Ablations: without the memory token or response pretraining, MuJoCo success drops to 54%; without gaze supervision, to 49.33%.
Limitations
Recovery is not always safe: four safety failures remain on nail insertion, and the authors name safer recovery and a longer memory horizon as future work. Adapted FTP-π0.5 still does better on knob operation (72% vs 66%). The simulation tasks are custom, with one task from a public benchmark.
Videos
Figures



Abstract
In precision manipulation, small errors in alignment or force can cause jamming or deformation, making contact feedback essential for adapting pretrained vision-language-action (VLA) models to these tasks. A single tactile observation leaves the response to recent actions ambiguous. Direct tactile fusion during post-training can also disperse the action expert's visual attention and reduce the share it allocates to contact regions. To address these challenges, CoRA-VTLA augments pretrained VLAs with lightweight tactile modules. A queryable action-response memory retrieves recent contact responses so that the action expert can combine them with immediate tactile feedback. Contact-guided gaze supervision concentrates the visual attention of the tactile semantic token on image regions corresponding to fingertip contact. CoRA-VTLA achieves 73.33% mean success across three MuJoCo tasks, compared with 50% for direct tactile fusion with the same sensor inputs. It also achieves 67% success on an optical-tactile task from the public UniVTAC benchmark and 66.7% mean success across three real-robot tasks. These results show that tactile post-training helps pretrained VLAs adjust to changing contact.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
