WiTac: Wrist-Informed Tactile Representations for Contact-Rich Manipulation
TL;DR WiTac is a pretrained encoder that fuses each finger's tactile history with wrist-camera views into one contact token per finger, then conditions a VLA policy. On four real bimanual dexterous-hand tasks it raises GR00T N1.7's average success from 37.5% to 66.3%, and π0.5's from 43.8% to 58.8%.
What they do
The same fingertip contact can call for different corrections depending on where the hand, object and target are; when seating a battery, similar finger loads can accompany different misalignments. WiTac fuses touch with wrist-camera vision before the policy sees it, producing ten finger-level contact tokens grounded in visual context. The encoder is pretrained on nearly 300 hours of SharpaDex v1.0 data, then frozen and plugged into vision-language-action (VLA) policies.
How it works
- Encoding. An 8-frame history of each finger’s 240x240 deformation map passes through a spatial and a causal temporal encoder. Each finger feature is modulated by its wrist-relative fingertip pose and by the robot state, then cross-attends to frozen DINOv3 patch features from both wrist cameras.
- Pretraining. Five self-supervised heads: reconstruct contact area, intensity and center; reconstruct deformation and fingertip-motion histories; recover masked wrist-image features, which forces the ten tokens to carry visual context; and predict fingertip wrench changes given future actions. No task-success labels are used.
- Policy injection. The frozen tokens enter selected action-expert blocks of GR00T N1.7 or π0.5 through gated cross-attention. The gate is supervised to open during contact.
Results
- Four real tasks absent from pretraining (battery insertion, bowl wiping, USB reinsertion, cabinet opening) on a robot with two Sharpa Wave hands; 200 demonstrations per task, 20 trials per method and task.
- With GR00T, average success is 66.3%, versus 37.5% for the vision-only base, 47.5% for wrist-free WiTac-NW, 50.0% for the VT-MUSE encoder, 40.0% for FTP-1 and 50.0% for T-Rex. On USB reinsertion, wrist-free touch lowers success from 10% to 5%, while WiTac raises it to 40%.
- With π0.5: 58.8%, versus 43.8% (base) and 51.3% (WiTac-NW).
- GR00T ablations: 53.8% without pretraining, 58.8% without masked visual recovery, 50.0% with touch and wrist features injected separately.
Limitations
Vision and touch are fused by feature-level attention, not aligned in 3D, and only the current wrist image is paired with the tactile history. Evaluation covers four tasks on one robot platform; transfer to other hands, tactile sensors and camera setups is untested.
Videos
Figures



Abstract
Similar tactile contacts can call for different corrections depending on the configuration of the hand, object and target. Rather than supplying touch and wrist vision separately, fusing them before policy injection better grounds tactile features in visual context, yielding more accurate contact representations. We introduce Wrist-Informed Tactile Representations (WiTac), a pretrained encoder that learns such representations before action generation. Pretrained on nearly 300 hours of SharpaDex data, WiTac deeply fuses tactile signals with hand motion and wrist views into a comprehensive contact representation. Conditioning downstream VLA policies (GR00T N1.7 and $π_{0.5}$) on WiTac substantially improves task success over vision-only base policies across four contact-rich manipulation tasks (+28.8% and +15.0% on average), and further outperforms a wrist-free tactile encoder by 18.8% and 7.5%, respectively. Offline analyses further show that incorporating wrist vision yields more accurate contact descriptors, stronger joint task-contact discrimination, greater feature stability under tactile noise, and clearer action-direction readout from similar contacts. Project page: https://wrist-informed-tactile-representations.github.io/.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
