Expressing Robot Incapability

Minae Kwon, Sandy H. Huang, Anca D. Dragan

Seminal October 2018

TL;DR When a robot cannot finish a task, it plays an optimized "attempt" in which its elbow, shoulder and base mimic the motion its stuck end-effector cannot make. In an online study with 120 people, 95% picked the robot's correct goal, against 60% when the robot simply repeated its failing action.

What they do

A robot that fails a task, such as lifting a cup that is too heavy or pushing a shelf that will not move, often does not get far enough for an observer to tell what it wanted or why it stopped. This paper generates an “attempt” motion that shows both. It is a trajectory as similar as possible to the successful one, subject to the constraint that the end-effector cannot move past the failure point.

How it works

Optimization. The attempt trades a similarity cost against smoothness and is solved with TrajOpt on a simulated PR2 in OpenRAVE.
Cost choice. They compare three costs: matching the configuration-space change, matching each body point’s own motion, and having the base, elbow and shoulder imitate the end-effector’s missing motion. They pair each with three distance metrics (squared L2, dot product, and a repeated-projection generalization of the dot product). Only end-effector imitation with the projection metric gave readable attempts on both lift and push.
Presentation. The robot plays the attempt fast, rewinds at moderate speed, and repeats three times. Both choices came from two 60-person online studies.

Results

Main study: 120 online participants watched videos of five tasks. The baseline robot repeatedly executes its failing action. With attempts, the correct goal’s average rank went from 1.65 to 1.05 (95% vs 60% picked it), with the largest gains on push and push sideways.
The correct cause of failure’s average rank went from 3.02 to 1.98.
Participants rated the robot more trustworthy and a better teammate, and were more willing to collaborate with it (all p<.0001).
Solving for an attempt takes about half a second with a fixed base and a few seconds when the base moves.

Limitations

The method only covers failures where the end-effector cannot reach a desired pose. It cannot express grasping or perception failures, and it does not consider pairing motion with speech or other channels. All motions were generated in simulation and shown as videos.

Videos

Figures

From Figure 1: The robot generates an attempt motion that resembles successful execution (moving the end-effector fro... — From Figure 1: The robot generates an attempt motion that resembles successful execution (moving the end-effector from the failure point to the desired pose) while obeying its limitations. Here it lifts its elbow to show it is trying to lift the cup, which is too heavy.

From Figure 2: The robot executes the task up to the point of failure (left), then plays the attempt trajectory (cent... — From Figure 2: The robot executes the task up to the point of failure (left), then plays the attempt trajectory (center), rewinds it (right), and repeats attempt and rewind two more times.

From Figure 5: Attempt trajectories for five incompletable tasks (lift, push, pull, pull down, push sideways). Arrows... — From Figure 5: Attempt trajectories for five incompletable tasks (lift, push, pull, pull down, push sideways). Arrows show how the elbow, shoulder and base move, imitating how the end-effector would move if the task succeeded.

From Figure 8: Participants' ratings of the correct goal for each task and averaged across tasks, for the optimizatio... — From Figure 8: Participants' ratings of the correct goal for each task and averaged across tasks, for the optimization-based attempts and the repeated-failure baseline.

Abstract

Our goal is to enable robots to express their incapability, and to do so in a way that communicates both what they are trying to accomplish and why they are unable to accomplish it. We frame this as a trajectory optimization problem: maximize the similarity between the motion expressing incapability and what would amount to successful task execution, while obeying the physical limits of the robot. We introduce and evaluate candidate similarity measures, and show that one in particular generalizes to a range of tasks, while producing expressive motions that are tailored to each task. Our user study supports that our approach automatically generates motions expressing incapability that communicate both what and why to end-users, and improve their overall perception of the robot and willingness to collaborate with it in the future.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong