Deep Haptic Model Predictive Control for Robot-Assisted Dressing

Zackory Erickson, Henry M. Clever, Greg Turk, C. Karen Liu, Charles C. Kemp

Seminal September 2017

TL;DR A PR2 pulls a hospital gown onto a person's arm with model predictive control, using LSTMs trained only in simulation to predict from end-effector force and motion how hard the garment will press on the arm. With a 0.2 s horizon it dressed the full arm in 98.75% of trials with 10 participants and avoided a sleeve catch on the fist in 97.5%.

What they do

When a robot dresses a person, the garment, not the gripper, presses on the body. This paper predicts those forces from the robot’s end-effector force/torque and motion, and uses model predictive control (MPC) to keep them low while a PR2 pulls a hospital gown sleeve onto a seated person’s arm.

How it works

Simulated data. 10,800 self-supervised dressing trials with random actions, from a cloth simulator whose garment parameters were fit to real PR2 data with CMA-ES. A force map records the forces on 37 points (taxels) across the fist, forearm and upper arm.
Two LSTMs. A predictor maps the current measurement and a proposed action (3D velocity plus yaw change) to the next 0.2 s of end-effector measurements; an estimator maps measurements to the force map. Feeding predictions back in gives longer horizons without retraining.
MPC. At 5 Hz the robot scores 28 fixed forward-facing candidate actions with a cost that penalizes predicted arm forces most, rewards forward progress and penalizes yaw. It runs on the PR2’s on-board CPUs.

Results

10 participants, 48 trials each, horizons of 0.01 s, 0.05 s and 0.2 s.
Full arm dressing success: 1.25%, 97.5% and 98.75%. With 0.01 s all candidates’ predicted force maps were nearly identical, so the robot pushed into the elbow until the 10 N stop. Longer horizons rotated around the elbow and pulled the gown to the shoulder, an emergent behavior.
Avoiding a sleeve catch on the fist: 6.25%, 26.25% and 97.5%. All horizons tried to lower the gripper, but only 0.2 s did so early enough.

Limitations

Participants held one fixed arm posture; predicting forces for other poses is left open. Haptics alone give no knowledge of where the person is, so the robot cannot recover if the garment misses the arm. Horizons are short, actions are held constant over a horizon, and backward motions were not allowed. Participants were able-bodied adults aged 18-30.

Figures

From Figure 1: A PR2 pulls a hospital gown onto a participant's arm by minimizing the predicted forces applied to the... — From Figure 1: A PR2 pulls a hospital gown onto a participant's arm by minimizing the predicted forces applied to the person's body.

From Figure 2: The physics-based simulation that generates the training data. It records force, torque, position and ... — From Figure 2: The physics-based simulation that generates the training data. It records force, torque, position and velocity at the end effector, plus a force map of everything the gown applies to the arm.

From Figure 3: The predictor LSTM turns the current measurement and a proposed action into future end-effector measur... — From Figure 3: The predictor LSTM turns the current measurement and a proposed action into future end-effector measurements, which the estimator LSTM maps to forces on the person's arm.

From Figure 10: Force at the robot's end effector during full arm dressing, averaged over 10 participants. With a 0.0... — From Figure 10: Force at the robot's end effector during full arm dressing, averaged over 10 participants. With a 0.01 s horizon the controller pushes into the elbow until the 10 N limit; 0.05 s and 0.2 s keep forces low.

Abstract

Robot-assisted dressing offers an opportunity to benefit the lives of many people with disabilities, such as some older adults. However, robots currently lack common sense about the physical implications of their actions on people. The physical implications of dressing are complicated by non-rigid garments, which can result in a robot indirectly applying high forces to a person's body. We present a deep recurrent model that, when given a proposed action by the robot, predicts the forces a garment will apply to a person's body. We also show that a robot can provide better dressing assistance by using this model with model predictive control. The predictions made by our model only use haptic and kinematic observations from the robot's end effector, which are readily attainable. Collecting training data from real world physical human-robot interaction can be time consuming, costly, and put people at risk. Instead, we train our predictive model using data collected in an entirely self-supervised fashion from a physics-based simulation. We evaluated our approach with a PR2 robot that attempted to pull a hospital gown onto the arms of 10 human participants. With a 0.2s prediction horizon, our controller succeeded at high rates and lowered applied force while navigating the garment around a persons fist and elbow without getting caught. Shorter prediction horizons resulted in significantly reduced performance with the sleeve catching on the participants' fists and elbows, demonstrating the value of our model's predictions. These behaviors of mitigating catches emerged from our deep predictive model and the controller objective function, which primarily penalizes high forces.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong