← Paper Digest

LuxTac: A Flexible, Scalable, and Open-Source Optical Taxel Array with Distributed Three-Axis Force Sensing for Tactile Robot Learning

Md Rakibul Islam Prince, John Pelan, Zhiyuan Zhang, Adyaa Khera, Katherine Pesetski, Adeesh Desai, Akimizu Sugiyama, Minwoo Ki, Benjamin Alan Capelle, Temitope Ibrahim Amosa, Zijie Xu, Yihan Huang, Taehun Chung, Long Wang, Wei Li, Shaoting Lin, Yu She

November 2026

Animation of one shear-cartridge taxel, with the light reaching its detector shown in the inset.

TL;DR A 3.5 mm thick, flexible tactile array whose taxels are off-the-shelf color light sensors under a silicone gel. A swappable gel with colored walls turns shear into a color change, so each taxel estimates three-axis force (per-taxel R² 0.92-0.94 on held-out sessions). Adding the pad's pressure readings to a visuomotor policy raised peg-in-hole success from 6/15 to 13/15.

What they do

Cheap piezoresistive tactile arrays sense pressure only. Camera-based sensors also resolve shear but are too bulky to tile over a gripper. LuxTac is a thin, flexible optical taxel array built from off-the-shelf parts. It senses pressure at each taxel and, with a second gel cartridge, three-axis force at each taxel, with no camera, magnets or bump overlay.

How it works

  • Taxel. Each taxel is a TCS34725 RGBC light sensor under 2 mm of translucent white silicone capped by a black film. LEDs light the gel, and pressing it brings the absorbing film closer, so less light returns. The 4x4 prototype streams at 100 Hz (up to about 180 Hz).
  • Shear cartridge. Red/green and yellow/blue silicone walls separate the taxels. Shear tilts the walls, so brightness tracks normal deformation and color tracks shear direction and size. A small per-taxel network maps each 3x3 neighborhood to force.
  • Simulation. The same measured signal-vs-depth curve drives a simulated pad in IsaacGym/TacSL.

Results

  • Normal cartridge: linear from 104 mN to at least 5.8 N (R² 0.999), with 2.0% repeatability and 5.9% hysteresis.
  • Shear cartridge on held-out sessions: per-taxel R² 0.92/0.93/0.94 in x/y/z and a median shear-direction error of 4.8°. Color alone beats brightness alone for shear.
  • Real and simulated grasp patterns agree with a mean cosine of 0.942.
  • Using pressure only (no shear), an ACT policy on a reBot arm raised peg-in-hole success from 6/15 (vision only) to 13/15. Cube sorting by surface texture barely changed (5/15 vs 6/15).

Limitations

Readings measure deformation, so force estimates depend on contact size; training must include the smallest expected contact. The silicone creeps under sustained load, and LED output drifts with temperature. The colored walls raise hysteresis to 9.1%. Shear was tested only with flat indenters and is not yet used as a policy input. The 4.4-4.8 mm pitch may be too coarse for texture.

Videos

Pressing the real pad by hand.
The per-taxel readout recorded during the presses in the clip above.

Figures

From Figure 3: Sensing principle. (a) Normal loading deforms the gel symmetrically and changes the light paths betwee...
From Figure 3: Sensing principle. (a) Normal loading deforms the gel symmetrically and changes the light paths between the LEDs and the photodetector. (b) Shear deforms it asymmetrically and changes the light the photodetector receives.
From Figure 2: Renderings of 8x8 and 8x4 arrays and the prototype board (top to bottom). Each taxel is a separate dig...
From Figure 2: Renderings of 8x8 and 8x4 arrays and the prototype board (top to bottom). Each taxel is a separate digital detector on its own bus segment, so the same cell tiles into other sizes without changing the readout. A US dime is shown for scale.
From Figure 7: Shear cartridge on held-out sessions. (a, c) Per-taxel normal and lateral force estimates vs. labels. ...
From Figure 7: Shear cartridge on held-out sessions. (a, c) Per-taxel normal and lateral force estimates vs. labels. (b, d) Summed estimates vs. the force/torque sensor. (e) R2 per indenter. (f) The model retrained on brightness only, color only, or both.
From Figure 9: Real-robot runs of peg-in-hole insertion (Task 1) and sorting smooth and dotted cubes (Task 2), from p...
From Figure 9: Real-robot runs of peg-in-hole insertion (Task 1) and sorting smooth and dotted cubes (Task 2), from positioning through grasping, transport, lowering and release, with the LuxTac heatmap at each stage.
Abstract

Low-cost tactile arrays have made touch a practical input for robot learning, but piezoresistive designs sense only pressure, while camera-based sensors that also resolve shear are too bulky to tile over a gripper. To solve this problem, we present LuxTac, a flexible, scalable optical taxel array, about 3.5 mm thick and built from off-the-shelf parts. Each taxel is a digital RGBC light-to-digital converter under a translucent white silicone layer capped by an opaque black film, so the back-scattered LED light falls monotonically with local indentation depth. The same board accepts two interchangeable gel cartridges: a plain one for normal sensing, and one with colored walls that encodes the local shear in the color of the returned light, with no camera, magnets, or bump overlay, making LuxTac, to our knowledge, the first optical taxel array to resolve shear at individual taxels. Characterized with an automated CNC indenter and a six-axis force/torque sensor, the normal cartridge responds linearly to force ($R^2$ = 0.999) from 104 mN to at least 5.8 N, with 2.0% repeatability, 5.9% hysteresis, and a taxel-to-taxel variation of 6-10% after normalization. With the shear cartridge, per-taxel models estimate the three-axis force at every taxel; across three indenter shapes and nine held-out sessions, the summed estimates reach $R^2$ = 0.96/0.96/0.92 in x/y/z, each taxel's estimate 0.92/0.93/0.94, and the median shear-direction error is 4.8 degrees. Adding LuxTac pressure readings to an Action Chunking Transformer policy substantially improves peg-in-hole success over a vision-only baseline. Design files, code, and videos are available at https://luxtac.github.io.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.