Grip, Trace, Write: Force-Aware Imitation and Reinforcement Learning for High-DoF Robotic Handwriting

Rongqian Chen, Yiding Chen, Jijie Zhou, Tom Z. Jiahao

November 2026

Top view: the hand writes the word EMBER over the gray target.

TL;DR A simulated 25-joint hand on a UR5e writes digits, letters, words and Chinese characters with a freely held pen, using a grasp policy, behavior cloning from inverse kinematics and RL on ink and force rewards. RL lifts mean stroke completion to 94.8% and cuts force error by 53% relative to behavior cloning. Simulation only.

What they do

Robot handwriting usually bolts the pen to the arm and tracks a path. Here the pen is a free rigid body held in a tripod grasp by a multi-fingered hand, so the fingers must keep hold of it while the arm and wrist move the tip and press with a requested force. The authors build a MuJoCo environment for this and train one policy that traces shapes and writes digits, capital letters, English words and Chinese characters.

How it works

Setup. A hand with 25 finger joints and a 2-joint wrist sits on a UR5e, controlled at 25 Hz. The policy sees 137 state entries (proprioception, tactile, task state) plus 16 upcoming trajectory samples giving tip displacement, height, requested force and time. No images.
Two-branch actor. One MLP outputs finger actions (grasp), another outputs arm, wrist and stop actions (writing). Both read the same state.
Three stages. The grasp branch is cloned from 240 teacher episodes, then trained with 8M PPO steps to hold the pen while still, moving and scribbling. An inverse-kinematics controller then writes with that grasp to produce demonstrations for behavior cloning. Finally PPO refines the writing branch with the grasp frozen.
Outcome reward. Ordered ink coverage within 1 mm, off-target ink, force error, elapsed time and pen drops; no term for imitating IK.

Results

Mean completion over eight tasks, 50 seeds each: 94.8% for RL, 91.3% for BC, 92.0% for IK. Ink precision rises from 79.0% (BC) to 87.6%.
Force error falls 53%, from 0.773 N to 0.362 N; time from 8.19 s to 7.67 s.
Zero-shot on unseen content, completion is 91.6% on English words, 92.9% on Chinese idioms and 91.6% on Chinese characters with seven or more strokes.
Final RL used 39.85M transitions in 193.8 minutes.

Limitations

Everything is in simulation; there is no real-robot test. The pen starts already seated in the grasp. The line task is the exception where IK (98.2%) beats RL (97.5%). RL training is non-monotone, and the learning curves are snapshots of a single run, not independent seeds.

Videos

Top view: writing a four-character Chinese idiom.

Side view: the arm and hand draw on the sheet with the freely held pen.

Figures

From Figure 3: Simulated setup. (a) A UR5e arm with a 2-DoF wrist and a 25-DoF anthropomorphic hand writes on paper w... — From Figure 3: Simulated setup. (a) A UR5e arm with a 2-DoF wrist and a 25-DoF anthropomorphic hand writes on paper with a free pen. (b) The tripod grasp. (c) Top view mid-episode; gray marks the remaining target.

From Figure 2: Training pipeline. Panels 1–3 correspond to the three stages: grasp BC and hold RL, all-task writing B... — From Figure 2: Training pipeline. Panels 1–3 correspond to the three stages: grasp BC and hold RL, all-task writing BC with the learned grasp, and full-task outcome RL on the composed policy.

From Figure 6: Matched episodes across IK, BC, and final RL (top), and their evolution from BC initialization through... — From Figure 6: Matched episodes across IK, BC, and final RL (top), and their evolution from BC initialization through RL (bottom). Blue is deposited ink, gray is the target, and red marks target points not covered within 1 mm. Numbers report completion and ink precision. The high-frequency deviations in IK and BC leave off-target ink; RL produces smoother, more target-aligned strokes.

Abstract

Most robotic handwriting rigidly mounts the pen and reduces writing to trajectory tracking. A freely held pen instead couples grasp stability, tip motion, and contact force. We present Grip, Trace, Write, a force-aware simulation environment and learning framework for high-DoF handwriting. A two-branch actor separates pen retention from arm-wrist writing while preserving their closed-loop coupling. Training learns a grasp controller, initializes writing from inverse-kinematics demonstrations by behavior cloning, and improves writing through reinforcement learning with rewards on ink, force, time, and retention. Across diverse tasks, the policy improves completion, precision, force regulation, and efficiency over behavior cloning and inverse kinematics, and generalizes without fine-tuning to unseen English and Chinese content. These results establish force-aware handwriting as a testbed for dexterous tool use.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong