Matt Strong

Matt Strong

CS PhD at Stanford University

← Paper Digest

Assistive Gym: A Physics Simulation Framework for Assistive Robotics

Zackory Erickson, Vamsee Gangaram, Ariel Kapusta, C. Karen Liu, Charles C. Kemp

★ Must read October 2019

TL;DR An open-source PyBullet framework with six simulated caregiving tasks, a human model with impairments and preferences, and PPO baselines for four commercial robots. Training the simulated person as an active partner raised the best robot's task success by 30.4 points on average.

What they do

Research on robots that physically help people is costly and risky to run with real people. Assistive Gym is an open-source simulation framework, built on PyBullet behind the OpenAI Gym interface, with six activities of daily living: itch scratching, bed bathing, drinking, feeding, dressing and arm manipulation. It includes the PR2, Jaco, Baxter and Sawyer robots, human models, and PPO baselines for every robot-task pair.

How it works

  • Human model. Male and female models with 50th-percentile sizes, weights and joint limits, 40 controllable joints and self-collision. Each environment randomly applies tremor, joint weakness or limited range of motion. A neural network trained on motion capture blocks arm poses outside pose-dependent joint limits.
  • Reward. The task reward is added to a human preference reward: weighted penalties, switched on per task, for fast end-effector motion, force away from the target, forces above 10 N near it, spilled food or water, food or water entering the mouth too fast, garment forces on the body and high pressure from large tools.
  • Setup. The robot base pose is the one of 100 samples with collision-free IK to the most goals. Actions are joint position changes; observations are end-effector pose and force, joint positions and positions of task-relevant human joints.
  • Training. PPO, 10M time steps per policy on 36 parallel actors, best of 3 seeds. In co-optimization a human policy and a robot policy are trained together on the same reward.

Results

  • With a static person, the best robot per task succeeded in 88% of feeding, 72% of drinking, 54% of itch-scratching, 26% of dressing, 24% of bed-bathing and 8% of arm-manipulation trials.
  • Robots differ: on itch scratching the Jaco scored a reward of 280.8 and the PR2 55.1, which the authors attribute to the PR2’s shorter arm span.
  • With an active, co-optimized person, best-robot success rose to 99% feeding, 89% dressing, 81% bed bathing, 79% drinking and 68% itch scratching.
  • Training took 2 h (itch scratching) to 19 h (drinking) on a 36-vCPU machine, and about 6 days for dressing because of cloth simulation.

Limitations

Simulation only; the authors say more work is needed before real robots benefit from these controllers. Dressing and arm manipulation stayed hard, and arm-manipulation policies often lifted the arm with the tool’s thin edge, applying high pressure. High forces are avoided by capping actuator strength rather than through learning. Arm manipulation was not evaluated with an active person.

Figures

From Figure 1: A Baxter robot giving dressing assistance, one of four robots and tasks shown in the teaser.
From Figure 1: A Baxter robot giving dressing assistance, one of four robots and tasks shown in the teaser.
From Figure 3: The person's physical state (e.g. force applied to them) shapes their mental state, which gives a huma...
From Figure 3: The person's physical state (e.g. force applied to them) shapes their mental state, which gives a human preference reward that is added to the robot's task success reward.
From Figure 5: A learned human policy has rotated the person's arm so the Jaco can scratch an itch underneath it.
From Figure 5: A learned human policy has rotated the person's arm so the Jaco can scratch an itch underneath it.
Abstract

Autonomous robots have the potential to serve as versatile caregivers that improve quality of life for millions of people worldwide. Yet, conducting research in this area presents numerous challenges, including the risks of physical interaction between people and robots. Physics simulations have been used to optimize and train robots for physical assistance, but have typically focused on a single task. In this paper, we present Assistive Gym, an open source physics simulation framework for assistive robots that models multiple tasks. It includes six simulated environments in which a robotic manipulator can attempt to assist a person with activities of daily living (ADLs): itch scratching, drinking, feeding, body manipulation, dressing, and bathing. Assistive Gym models a person's physical capabilities and preferences for assistance, which are used to provide a reward function. We present baseline policies trained using reinforcement learning for four different commercial robots in the six environments. We demonstrate that modeling human motion results in better assistance and we compare the performance of different robots. Overall, we show that Assistive Gym is a promising tool for assistive robotics research.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.