Trust-Aware Decision Making for Human-Robot Collaboration: Model Learning and Planning
TL;DR A robot learns a POMDP with human trust as a hidden state from online user data, then orders its actions to build or lower that trust. In a real-robot table-clearing study with 20 participants, human interventions fell to zero on the can and 71% on the wine glass compared with a policy that ignores trust.
What they do
The authors make human trust a latent variable in a POMDP, so the robot infers it from the human’s actions, predicts how its own actions change it, and plans for long-term team reward. The test is clearing three water bottles, a fish can and a wine glass from a table: as the robot reaches for each object, the human either stays put or intervenes and moves it themselves.
How it works
- Rewards. Robot success earns 1/2/3 for bottle/can/glass, robot failure 0/-4/-9, and a human intervention 0.
- Trust dynamics. Trust is a scalar from 1 to 7 (Muir’s questionnaire). A linear Gaussian model maps current trust and the outcome (success, failure or intervention, per object) to the next trust level, fit with Hamiltonian Monte Carlo on data from 222 Mechanical Turk participants.
- Human model. A sigmoid choice between staying put and intervening, where the human’s belief in robot success is linear in trust. It fits better than a trust-free model (log-likelihood -153.37 vs -156.40).
- Planning. The state is the remaining objects plus trust; SARSOP solves the POMDP. The myopic baseline uses the trust-free human model.
Results
- The trust-POMDP clears the bottles first and the glass last; the myopic robot goes glass, can, then bottles. Team reward was significantly higher online (201 participants, p=0.006) and with the real robot (20 participants, p=0.004).
- Online, interventions fell 54% on the can and 31% on the glass; on the real robot, to zero on the can and 71% lower on the glass.
- In simulation, with a 0.9 failure probability on the glass, the robot probes trust with the can; if the human stays put, it fails on purpose on the bottles so the human will take the glass. Over 10^4 runs this averaged -1.36 reward versus -1.65 for a trust-maximizing policy (p<0.001).
Limitations
In the human studies the robot always succeeded; failures were studied only in simulation with the learned model. Trust is one scalar for the whole system, robot capabilities are assumed static and the reward is hand-specified. Average reported trust did not differ significantly between conditions.
Figures
Abstract
Trust in autonomy is essential for effective human-robot collaboration and user adoption of autonomous systems such as robot assistants. This paper introduces a computational model which integrates trust into robot decision-making. Specifically, we learn from data a partially observable Markov decision process (POMDP) with human trust as a latent variable. The trust-POMDP model provides a principled approach for the robot to (i) infer the trust of a human teammate through interaction, (ii) reason about the effect of its own actions on human trust, and (iii) choose actions that maximize team performance over the long term. We validated the model through human subject experiments on a table-clearing task in simulation (201 participants) and with a real robot (20 participants). In our studies, the robot builds human trust by manipulating low-risk objects first. Interestingly, the robot sometimes fails intentionally in order to modulate human trust and achieve the best team performance. These results show that the trust-POMDP calibrates trust to improve human-robot team performance over the long term. Further, they highlight that maximizing trust alone does not always lead to the best performance.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.