LuxTac: A Flexible, Scalable, and Open-Source Optical Taxel Array with Distributed Three-Axis Force Sensing for Tactile Robot Learning
TL;DR A 3.5 mm thick, flexible tactile array whose taxels are off-the-shelf color light sensors under a silicone gel. A swappable gel with colored walls turns shear into a color change, so each taxel estimates three-axis force (per-taxel R² 0.92-0.94 on held-out sessions). Adding the pad's pressure readings to a visuomotor policy raised peg-in-hole success from 6/15 to 13/15.
What they do
Cheap piezoresistive tactile arrays sense pressure only. Camera-based sensors also resolve shear but are too bulky to tile over a gripper. LuxTac is a thin, flexible optical taxel array built from off-the-shelf parts. It senses pressure at each taxel and, with a second gel cartridge, three-axis force at each taxel, with no camera, magnets or bump overlay.
How it works
- Taxel. Each taxel is a TCS34725 RGBC light sensor under 2 mm of translucent white silicone capped by a black film. LEDs light the gel, and pressing it brings the absorbing film closer, so less light returns. The 4x4 prototype streams at 100 Hz (up to about 180 Hz).
- Shear cartridge. Red/green and yellow/blue silicone walls separate the taxels. Shear tilts the walls, so brightness tracks normal deformation and color tracks shear direction and size. A small per-taxel network maps each 3x3 neighborhood to force.
- Simulation. The same measured signal-vs-depth curve drives a simulated pad in IsaacGym/TacSL.
Results
- Normal cartridge: linear from 104 mN to at least 5.8 N (R² 0.999), with 2.0% repeatability and 5.9% hysteresis.
- Shear cartridge on held-out sessions: per-taxel R² 0.92/0.93/0.94 in x/y/z and a median shear-direction error of 4.8°. Color alone beats brightness alone for shear.
- Real and simulated grasp patterns agree with a mean cosine of 0.942.
- Using pressure only (no shear), an ACT policy on a reBot arm raised peg-in-hole success from 6/15 (vision only) to 13/15. Cube sorting by surface texture barely changed (5/15 vs 6/15).
Limitations
Readings measure deformation, so force estimates depend on contact size; training must include the smallest expected contact. The silicone creeps under sustained load, and LED output drifts with temperature. The colored walls raise hysteresis to 9.1%. Shear was tested only with flat indenters and is not yet used as a policy input. The 4.4-4.8 mm pitch may be too coarse for texture.
Videos
Figures




Abstract
Low-cost tactile arrays have made touch a practical input for robot learning, but piezoresistive designs sense only pressure, while camera-based sensors that also resolve shear are too bulky to tile over a gripper. To solve this problem, we present LuxTac, a flexible, scalable optical taxel array, about 3.5 mm thick and built from off-the-shelf parts. Each taxel is a digital RGBC light-to-digital converter under a translucent white silicone layer capped by an opaque black film, so the back-scattered LED light falls monotonically with local indentation depth. The same board accepts two interchangeable gel cartridges: a plain one for normal sensing, and one with colored walls that encodes the local shear in the color of the returned light, with no camera, magnets, or bump overlay, making LuxTac, to our knowledge, the first optical taxel array to resolve shear at individual taxels. Characterized with an automated CNC indenter and a six-axis force/torque sensor, the normal cartridge responds linearly to force ($R^2$ = 0.999) from 104 mN to at least 5.8 N, with 2.0% repeatability, 5.9% hysteresis, and a taxel-to-taxel variation of 6-10% after normalization. With the shear cartridge, per-taxel models estimate the three-axis force at every taxel; across three indenter shapes and nine held-out sessions, the summed estimates reach $R^2$ = 0.96/0.96/0.92 in x/y/z, each taxel's estimate 0.92/0.93/0.94, and the median shear-direction error is 4.8 degrees. Adding LuxTac pressure readings to an Action Chunking Transformer policy substantially improves peg-in-hole success over a vision-only baseline. Design files, code, and videos are available at https://luxtac.github.io.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
