Getting the Ball Rolling: Learning a Dexterous Policy for a Biomimetic Tendon-Driven Hand with Rolling Contact Joints

Yasunori Toshimitsu, Benedek Forrai, Barnabas Gavin Cangan, Ulrich Steger, Manuel Knecht, Stefan Weirich, Robert K. Katzschmann

Seminal August 2023

The trained policy rotating a sphere in the real Faive Hand.

TL;DR Introduces the Faive Hand, a 3D-printed tendon-driven hand with rolling contact joints, and models those joints in IsaacGym to train an RL policy. After about one hour of training on a single GPU, the policy transfers zero-shot to the real hand and rotates a sphere in-hand using only proprioception.

What they do

The paper introduces the Faive Hand (prototype “Proto 0”), a biomimetic tendon-driven hand built from 3D-printed parts and servo motors, meant as an accessible platform for dexterous manipulation. Its fingers use rolling contact joints, which have no fixed axis of rotation and no joint encoders. The authors model these joints in IsaacGym, train an RL policy for in-hand sphere rotation, and run it closed-loop on the physical hand.

How it works

Hand. 11 actuated degrees of freedom (3 in the thumb, 2 per other finger) and 16 joints, driven by 16 Dynamixel servos. Each rolling contact joint is two curved surfaces held together by crosswise ligament strings.
Simulation model. Each rolling contact joint is modelled as two virtual hinge joints, constrained to rotate together with fixed tendon elements in the MJCF file.
Low-level control. An analytic map from joint angles to tendon lengths, derived from CAD, turns joint commands into motor commands. An extended Kalman filter estimates joint angles back from tendon lengths.
Policy. PPO with asymmetric actor and critic MLPs, 4096 parallel environments, 20 Hz control, domain randomization of physics properties. The actor sees only joint position history, commands and previous actions, not the object.

Results

All policies trained in about one hour on a single NVIDIA A10G GPU.
On the real hand, the policy trained with domain randomization rotates the ball consistently and stays in the target velocity range (1 to 3 rad/s) for the majority of measurements, in both rotation directions. The policy without it only rocks the ball back and forth.
The hand held payloads of up to 10 kg in a five-finger power grasp, at a system weight of 1.1 kg.

Limitations

Only rotation about one axis was learned; the other two axes failed, which the authors attribute to the missing abduction and adduction joints. Real-robot performance varied strongly across random seeds, so the best seed per condition was reported, and joint position measurements had to be scaled by 0.5. Cube reorientation worked in simulation but failed on the real robot, attributed to poor EKF joint estimates under contact and missing system identification.

Videos

A second real-robot run of the in-hand sphere rotation policy.

Parallel IsaacGym environments used to train the sphere rotation policy.

Figures

From Figure 1: (a) The GPU-based parallelized simulation environment simulating 4096 robot hands in parallel to train... — From Figure 1: (a) The GPU-based parallelized simulation environment simulating 4096 robot hands in parallel to train an RL policy. (b) The trained policy deployed on the tendon-driven robot hand with rolling contact joints.

From Figure 2: (a) The Faive Hand has a rolling contact joint design with tendons and ligaments mimicking that of a h... — From Figure 2: (a) The Faive Hand has a rolling contact joint design with tendons and ligaments mimicking that of a human finger. (b) Overview of each component of the Faive Hand.

From Figure 4: Overview of the RL training framework. After training the policy within a simulation environment, the ... — From Figure 4: Overview of the RL training framework. After training the policy within a simulation environment, the actor network is transferred to the real robot.

From Figure 6: Distribution of the object rotational velocity on the real and simulated robot, for policies trained w... — From Figure 6: Distribution of the object rotational velocity on the real and simulated robot, for policies trained with and without DR. The gray strip indicates the region in which the object rotation reward is at its maximum value. The diamond indicates the mean.

Abstract

Biomimetic, dexterous robotic hands have the potential to replicate much of the tasks that a human can do, and to achieve status as a general manipulation platform. Recent advances in reinforcement learning (RL) frameworks have achieved remarkable performance in quadrupedal locomotion and dexterous manipulation tasks. Combined with GPU-based highly parallelized simulations capable of simulating thousands of robots in parallel, RL-based controllers have become more scalable and approachable. However, in order to bring RL-trained policies to the real world, we require training frameworks that output policies that can work with physical actuators and sensors as well as a hardware platform that can be manufactured with accessible materials yet is robust enough to run interactive policies. This work introduces the biomimetic tendon-driven Faive Hand and its system architecture, which uses tendon-driven rolling contact joints to achieve a 3D printable, robust high-DoF hand design. We model each element of the hand and integrate it into a GPU simulation environment to train a policy with RL, and achieve zero-shot transfer of a dexterous in-hand sphere rotation skill to the physical robot hand.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong