← Paper Digest

WiTac: Wrist-Informed Tactile Representations for Contact-Rich Manipulation

Haotian He, Yiyang Chen, Congsheng Xu, Wenchao Yue, Mingze Ye, Shuai Chen, Yi Yang, Yaping Li, Haonan Wang, Weinan E, Wenzhao Lian, Zhijie Deng

November 2026

Real-robot rollouts with two dexterous hands, starting with battery insertion.

TL;DR WiTac is a pretrained encoder that fuses each finger's tactile history with wrist-camera views into one contact token per finger, then conditions a VLA policy. On four real bimanual dexterous-hand tasks it raises GR00T N1.7's average success from 37.5% to 66.3%, and π0.5's from 43.8% to 58.8%.

What they do

The same fingertip contact can call for different corrections depending on where the hand, object and target are; when seating a battery, similar finger loads can accompany different misalignments. WiTac fuses touch with wrist-camera vision before the policy sees it, producing ten finger-level contact tokens grounded in visual context. The encoder is pretrained on nearly 300 hours of SharpaDex v1.0 data, then frozen and plugged into vision-language-action (VLA) policies.

How it works

  • Encoding. An 8-frame history of each finger’s 240x240 deformation map passes through a spatial and a causal temporal encoder. Each finger feature is modulated by its wrist-relative fingertip pose and by the robot state, then cross-attends to frozen DINOv3 patch features from both wrist cameras.
  • Pretraining. Five self-supervised heads: reconstruct contact area, intensity and center; reconstruct deformation and fingertip-motion histories; recover masked wrist-image features, which forces the ten tokens to carry visual context; and predict fingertip wrench changes given future actions. No task-success labels are used.
  • Policy injection. The frozen tokens enter selected action-expert blocks of GR00T N1.7 or π0.5 through gated cross-attention. The gate is supervised to open during contact.

Results

  • Four real tasks absent from pretraining (battery insertion, bowl wiping, USB reinsertion, cabinet opening) on a robot with two Sharpa Wave hands; 200 demonstrations per task, 20 trials per method and task.
  • With GR00T, average success is 66.3%, versus 37.5% for the vision-only base, 47.5% for wrist-free WiTac-NW, 50.0% for the VT-MUSE encoder, 40.0% for FTP-1 and 50.0% for T-Rex. On USB reinsertion, wrist-free touch lowers success from 10% to 5%, while WiTac raises it to 40%.
  • With π0.5: 58.8%, versus 43.8% (base) and 51.3% (WiTac-NW).
  • GR00T ablations: 53.8% without pretraining, 58.8% without masked visual recovery, 50.0% with touch and wrist features injected separately.

Limitations

Vision and touch are fused by feature-level attention, not aligned in 3D, and only the current wrist image is paired with the tactile history. Evaluation covers four tasks on one robot platform; transfer to other hands, tactile sensors and camera setups is untested.

Videos

More real-robot rollouts, including wiping the inside of a bowl with a sponge.
Measured fingertip deformation next to WiTac's reconstruction during USB reinsertion.

Figures

From Figure 1: Left, WiTac is pretrained on SharpaDex v1.0 and conditions a downstream policy; the battery-insertion ...
From Figure 1: Left, WiTac is pretrained on SharpaDex v1.0 and conditions a downstream policy; the battery-insertion cartoon shows how wrist context resolves alignment ambiguity during contact. Right, the evaluation tasks and their success rates.
From Figure 2: (a) State-conditioned tactile histories query wrist-camera features to form contact tokens. (b) Pretra...
From Figure 2: (a) State-conditioned tactile histories query wrist-camera features to form contact tokens. (b) Pretraining-only heads supervise contact state, deformation and fingertip-motion histories, masked visual recovery and action-conditioned wrench responses. (c) Tokens from the frozen encoder condition the action expert through gated cross-attention.
From Figure 3: Rollouts from matched initial conditions. The wrist-free WiTac-NW rotates the battery out of alignment...
From Figure 3: Rollouts from matched initial conditions. The wrist-free WiTac-NW rotates the battery out of alignment and leaves the USB connector inserted, whereas WiTac seats the battery and extracts the connector.
Abstract

Similar tactile contacts can call for different corrections depending on the configuration of the hand, object and target. Rather than supplying touch and wrist vision separately, fusing them before policy injection better grounds tactile features in visual context, yielding more accurate contact representations. We introduce Wrist-Informed Tactile Representations (WiTac), a pretrained encoder that learns such representations before action generation. Pretrained on nearly 300 hours of SharpaDex data, WiTac deeply fuses tactile signals with hand motion and wrist views into a comprehensive contact representation. Conditioning downstream VLA policies (GR00T N1.7 and $π_{0.5}$) on WiTac substantially improves task success over vision-only base policies across four contact-rich manipulation tasks (+28.8% and +15.0% on average), and further outperforms a wrist-free tactile encoder by 18.8% and 7.5%, respectively. Offline analyses further show that incorporating wrist vision yields more accurate contact descriptors, stronger joint task-contact discrimination, greater feature stability under tactile noise, and clearer action-direction readout from similar contacts. Project page: https://wrist-informed-tactile-representations.github.io/.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.