TacSL: A Library for Visuotactile Sensor Simulation and Learning

Iretiayo Akinola, Jie Xu, Jan Carius, Dieter Fox, Yashraj Narang

Seminal August 2024

The tactile policy places a peg upright on a pedestal on the real robot.

TL;DR TacSL simulates GelSight tactile images and contact force fields on the GPU inside Isaac Simulator, over 200x faster than prior tactile simulators, and adds a policy-learning toolkit with a new RL method (AACD). Tactile policies trained in simulation transfer zero-shot to a real robot, succeeding 91.4% of the time on peg placement and 82.7% on peg insertion.

What they do

Visuotactile sensors such as GelSight are expensive to simulate (contact, membrane deformation, lighting, imaging), so tactile policies have rarely been trained with online RL in simulation. TacSL is a GPU tactile simulation module inside Isaac Simulator that outputs both tactile RGB images and normal/shear force fields. It also ships two GelSight sensor models, three assembly tasks and learning algorithms.

How it works

Contact. Membrane and object are rigid bodies joined by a penalty-based spring-damper (Kelvin-Voigt) soft contact, solved as an implicit spring in PhysX for stability.
Tactile signals. A camera inside the sensor renders depth, mapped to RGB by a calibrated look-up table, all on the GPU with no external renderer. Force fields come from querying the object’s SDF at sampled tactile points, in parallel and for arbitrary meshes.
Learning. BC, DAgger and asymmetric actor-critic RL from a privileged-state expert. The new AACD initializes the tactile agent’s critic with the pretrained low-dimensional expert critic, then trains the image-based actor with PPO.
Sim-to-real. Randomized contact stiffness and damping, per-episode spatial and color augmentation of tactile images, and two-stage training (privileged expert, then a tactile student).

Results

Speed: 1631 FPS for tactile images with 512 environments on an RTX 3090, against 7.28 FPS for Taxim on one CPU core. 10x10 force fields run at 1.54M FPS with 32768 environments, 428x faster than the CPU baseline.
Simulated insertion: a reduced, real-world-available state reaches at most 8.7% success; adding tactile images gives 82-91%. With full image augmentation, plain asymmetric actor-critic fails to learn while AACD succeeds.
Real robot, zero-shot, 81 trials each: peg placement succeeds 87.7% with color augmentation against 27.2% without, and 91.4% with difference images. Peg insertion succeeds 82.7%. Both policies react to a person pushing the peg in the gripper.

Limitations

The contact law is linear and the depth-to-RGB mapping is a look-up table, which may be less accurate than a learned model, especially for curved sensors. Only tactile-image policies were transferred to the real robot, not force-field ones. The bolt-on-nut task is left as a challenge with no reported results.

Videos

Peg placement while a person pushes the peg in the gripper, 2x speed.

Real-robot peg insertion, first with the lights on and then in the dark.

Figures

From Figure 1: TacSL gives a simulated robot (left) the same tactile sensing as the real robot (right): a GelSight se... — From Figure 1: TacSL gives a simulated robot (left) the same tactile sensing as the real robot (right): a GelSight sensor on each finger producing a tactile RGB image and a tactile force field. Policies for tasks such as peg insertion are trained in simulation and transferred to the real system.

From Figure 2: The three parts of TacSL: a fast visuotactile simulation module that produces tactile images and force... — From Figure 2: The three parts of TacSL: a fast visuotactile simulation module that produces tactile images and force fields (left), a set of sensors and manipulation tasks (top right), and offline and online distillation plus RL algorithms for sim-to-real transfer (bottom right).

From Figure 10: Training curves for a tactile-image policy with the full image augmentation needed for sim-to-real tr... — From Figure 10: Training curves for a tactile-image policy with the full image augmentation needed for sim-to-real transfer. AAC with a randomly initialized critic (orange) never succeeds. AACD with a frozen (purple) or unfrozen (blue) pretrained critic learns the task, and the unfrozen critic reaches the higher success rate.

From Figure 12: Zero-shot peg-placement success on the real robot at three placement locations and on average, for fo... — From Figure 12: Zero-shot peg-placement success on the real robot at three placement locations and on average, for four tactile inputs: Vanilla (raw RGB, no image augmentation), ColorAug (raw RGB with augmentation), Diff+ColorAug (difference from the no-contact image) and Concat+ColorAug (current and no-contact images stacked). Results are consistent across locations.

Abstract

For both humans and robots, the sense of touch, known as tactile sensing, is critical for performing contact-rich manipulation tasks. Three key challenges in robotic tactile sensing are 1) interpreting sensor signals, 2) generating sensor signals in novel scenarios, and 3) learning sensor-based policies. For visuotactile sensors, interpretation has been facilitated by their close relationship with vision sensors (e.g., RGB cameras). However, generation is still difficult, as visuotactile sensors typically involve contact, deformation, illumination, and imaging, all of which are expensive to simulate; in turn, policy learning has been challenging, as simulation cannot be leveraged for large-scale data collection. We present TacSL (taxel), a library for GPU-based visuotactile sensor simulation and learning. TacSL can be used to simulate visuotactile images and extract contact-force distributions over $200\times$ faster than the prior state-of-the-art, all within the widely-used Isaac Simulator. Furthermore, TacSL provides a learning toolkit containing multiple sensor models, contact-intensive training environments, and online/offline algorithms that can facilitate policy learning for sim-to-real applications. On the algorithmic side, we introduce a novel online reinforcement-learning algorithm called asymmetric actor-critic distillation (AACD), designed to effectively and efficiently learn tactile-based policies in simulation that can transfer to the real world. Finally, we demonstrate the utility of our library and algorithms by evaluating the benefits of distillation and multimodal sensing for contact-rich manipulation tasks, and most critically, performing sim-to-real transfer. Supplementary videos and results are at https://iakinola23.github.io/tacsl/.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong