A System for General In-Hand Object Re-Orientation

Tao Chen, Jie Xu, Pulkit Agrawal

★ Must read November 2021

In simulation, the downward-facing hand turns a cup in mid-air to match the goal pose on the left.

TL;DR Model-free RL with teacher-student distillation trains a simulated Shadow Hand to reorient objects to any 3D orientation, palm up or palm down, without knowing their shape. On more than 2000 EGAD objects it succeeds 96% of the time palm up and over 80% palm down in mid-air.

What they do

They train a multi-finger hand to rotate an object from any orientation to any goal orientation. The hand faces up, faces down over a table, or faces down with nothing below it. Everything runs on a simulated 24-DoF Shadow Hand in Isaac Gym. The objects are 2282 EGAD and 78 YCB meshes, with 5 random scalings of each. The policies never see the object’s shape.

How it works

Teacher. PPO with 40K parallel environments trains an MLP or GRU policy on privileged state: joint, fingertip and object poses and velocities, plus the goal. It outputs relative joint-target changes at 60 Hz, with domain randomization and observation/action noise.
Students. DAgger distills the teacher into policies that use only real-world-style inputs. One sees joint positions, object position and the rotation to the goal. The other sees a voxelized point cloud from two RGBD cameras plus a goal point cloud, encoded by a sparse 3D IMPALA network.
Palm down, in air. Training this case directly gave 0% success. Episodes instead start from the end states of a separately trained lifting policy. For YCB objects, a gravity curriculum also steps gravity from +1 down to -9.81 m/s².

Results

Palm up, RNN teacher: 95.95% on EGAD and 80.40% on YCB. The reduced-state student reaches 91.96% and 81.04%. The palm-up policy learned on its own to toss the object with a spin and catch it at the goal orientation.
Palm down over a table: 95.31% on EGAD and 81.59% on YCB.
Palm down in mid-air: 82.27% on EGAD. The gravity curriculum lifts YCB from 52.81% to 74.74%.
Vision students, trained one per object: over 80% on the non-symmetric toy and mug, and over 50% on symmetric objects.

Limitations

All results are in simulation; real-robot transfer is argued from domain randomization and torque analysis, not tested. Each vision policy covers a single object, because GPU memory is the bottleneck. Long, small or thin objects cause 60% of palm-up YCB failures. The authors say they cannot fully rule out that the throwing strategy exploits the simulator.

Videos

The same mid-air task with a can and no table underneath.

With a table below, the downward-facing hand picks up a soup can and reorients it.

Figures

From Figure 1: Reorientation rollouts with the hand facing upward (top), facing downward over a table (middle) and fa... — From Figure 1: Reorientation rollouts with the hand facing upward (top), facing downward over a table (middle) and facing downward with no support (bottom). The rightmost image in each row shows the target orientation.

From Figure 2: The vision policy. A sparse 3D convolutional network (MK: Minkowski Engine) encodes the scene and goal... — From Figure 2: The vision policy. A sparse 3D convolutional network (MK: Minkowski Engine) encodes the scene and goal point cloud. Its features are joined with the hand joint positions and previous action, then a GRU outputs the next action.

Abstract

In-hand object reorientation has been a challenging problem in robotics due to high dimensional actuation space and the frequent change in contact state between the fingers and the objects. We present a simple model-free framework that can learn to reorient objects with both the hand facing upwards and downwards. We demonstrate the capability of reorienting over 2000 geometrically different objects in both cases. The learned policies show strong zero-shot transfer performance on new objects. We provide evidence that these policies are amenable to real-world operation by distilling them to use observations easily available in the real world. The videos of the learned policies are available at: https://taochenshh.github.io/projects/in-hand-reorientation.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong