In-Hand Object Rotation via Rapid Motor Adaptation

Haozhi Qi, Ashish Kumar, Roberto Calandra, Yi Ma, Jitendra Malik

★ Must read October 2022

Rotating a tennis ball with two estimated extrinsics plotted live as objects are swapped, partly at 5x speed.

TL;DR Train a fingertip-only object-rotation policy in simulation on cylinders, plus a module that infers object properties from 1.5 s of joint history. Deployed without fine-tuning on an Allegro hand, it rotates over 30 real objects of 4.5-7.5 cm and 5-200 g, averaging 23.96 rad per 30 s on heavy objects.

What they do

The task is rotating an object about the z-axis using only the fingertips of a 16-DoF Allegro hand, with no vision or touch. The policy is trained entirely in simulation on cylinders and runs on the real hand without fine-tuning. Borrowing rapid motor adaptation from legged locomotion, the policy is conditioned on an 8-dim “extrinsics” vector describing the object, which is estimated online from proprioception at deployment.

How it works

Base policy. An MLP takes the last three joint positions and actions plus the extrinsics and outputs PD targets at 20 Hz. An encoder compresses privileged object properties (position, size, mass, friction, center of mass) into the extrinsics. Both are trained with PPO in IsaacGym on 16,384 parallel environments with randomized scale, mass, friction and disturbance forces.
Reward. Clipped rotation velocity, plus penalties on deviation from the initial hand pose, torque, mechanical work and object linear velocity. No finger-gait heuristic is used; a stable gait emerges from the pose and energy terms.
Adaptation module. A temporal CNN over 30 steps (1.5 s) of joint positions and actions regresses the extrinsics. It is trained with supervised learning in simulation and replaces the encoder at deployment.

Results

Simulation, out-of-distribution objects: rotation reward 160.6, against 140.8 for domain randomization, 132.6 for explicit system identification and 165.1 for the privileged expert.
Real objects over 100 g: 23.96 rad rotated on average per 30 s episode (about 0.8 rad/s), better than all baselines on every metric. The baseline that freezes its first estimate does not transfer.
Across 33 real objects, rotation is almost perfectly stable and continuous on 22. When objects are swapped mid-run, individual extrinsics track object diameter and mass.

Limitations

Only single-axis rotation is evaluated; a ±z policy is shown qualitatively in the appendix. Most real failures come from wrong contact points, which proprioception cannot sense. Objects under 4 cm make the fingers collide, and extreme shapes are harder. No real-world data is used to improve the policy.

Videos

Tennis ball and toy cube rotated in side-by-side runs, including pushes from a human hand.

Multi-axis policy switching between +z and -z rotation on a polyhedral toy.

Figures

From Figure 1: Left, the controller is trained only in simulation on cylinders of different sizes and weights. Right,... — From Figure 1: Left, the controller is trained only in simulation on cylinders of different sizes and weights. Right, it is deployed without real-world fine-tuning on diverse objects, each labeled with its mass and shortest/longest diameter along the fingertips.

From Figure 2: Top, the base policy and object-property encoder are trained jointly with PPO. Middle, with the policy... — From Figure 2: Top, the base policy and object-property encoder are trained jointly with PPO. Middle, with the policy frozen, an adaptation module learns to estimate the extrinsics from proprioception and action history. Bottom, at deployment the policy uses the extrinsics estimated online by the adaptation module.

From Figure 7: Mean commanded torque against object mass for the real-world objects; the policy commands smaller torq... — From Figure 7: Mean commanded torque against object mass for the real-world objects; the policy commands smaller torque for lighter objects.

Abstract

Generalized in-hand manipulation has long been an unsolved challenge of robotics. As a small step towards this grand goal, we demonstrate how to design and learn a simple adaptive controller to achieve in-hand object rotation using only fingertips. The controller is trained entirely in simulation on only cylindrical objects, which then - without any fine-tuning - can be directly deployed to a real robot hand to rotate dozens of objects with diverse sizes, shapes, and weights over the z-axis. This is achieved via rapid online adaptation of the controller to the object properties using only proprioception history. Furthermore, natural and stable finger gaits automatically emerge from training the control policy via reinforcement learning. Code and more videos are available at https://haozhi.io/hora

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong