Preference-Based Learning for Exoskeleton Gait Optimization

Maegan Tucker, Ellen Novoseller, Claudia Kann, Yanan Sui, Yisong Yue, Joel Burdick, Aaron D. Ames

Seminal September 2019

TL;DR CoSpar picks an exoskeleton's gait from a library of precomputed stable gaits using only pairwise preferences ("better than the last trial?") and suggestions like "longer step". In 20 trials per user on the Atalante exoskeleton, its learned preferences matched the blind rankings of all three subjects.

What they do

The Atalante exoskeleton walks without crutches using gaits generated by partial hybrid zero dynamics, but nothing in that optimization captures user comfort, and users struggle to give reliable numerical scores. CoSpar instead learns which gait in a precomputed gait library a user prefers, from pairwise comparisons and suggested improvements.

How it works

Utility model. A preference-based Gaussian process over the finite set of gaits, with the posterior estimated by a Laplace approximation.
Gait selection. Extends the Self-Sparring dueling-bandit algorithm: draw n samples from the posterior and run the best gait under each. The user compares these with each other and with the b trials before them.
Coactive feedback. The user can also suggest a change (e.g. a longer step). The suggested gait is added to the data as preferred over the tested one, with a weight that can be set lower than pairwise preferences.

Results

Simulation: on a compass-gait biped, preferences derived from cost of transport make the posterior peak at the lowest-cost step length. On 100 random 2D functions (30x30 grid, 150 evaluations), adding coactive feedback improved learning in every (n, b) setting.
Step length (15 values, 0.08-0.18 m), three able-bodied subjects, 20 trials each with n=1, b=1: after five trials CoSpar had found a compact preferred region. Blind rankings of the posterior’s maximum, mean and minimum gaits matched the posterior for every subject.
Two features at once (step length with duration, and with width) for one subject: estimated preferences were consistent with a 3-gait blind ranking.
Metabolic expenditure was recorded but did not reflect preferences, since users do not have to exert effort to walk.

Limitations

CoSpar only selects from a precomputed gait library over one or two discretized features. Larger parameter sets would likely need high-dimensional learning methods, and generating new gaits is left to future work. Human tests cover three able-bodied subjects, with the 2D experiments run on one of them.

Figures

From Figure 1: The Atalante exoskeleton with and without a user, who wears a mask that measures metabolic expenditure.

From Figure 2: Left, the compass-gait biped's cost of transport over step length at 0.2 m/s. Right, CoSpar's posterio... — From Figure 2: Left, the compass-gait biped's cost of transport over step length at 0.2 m/s. Right, CoSpar's posterior utility (mean +/- 2 std, with 3 samples) after 0, 5, 20 and 75 preferences, converging on the optimal step length.

From Figure 4: Normalized objective value reached on 100 synthetic 2D functions, with and without coactive feedback, ... — From Figure 4: Normalized objective value reached on 100 synthetic 2D functions, with and without coactive feedback, for three (n, b) settings. Coactive feedback always helps, and n=2, b=0, which gets the fewest preferences, performs worst.

From Figure 5: Step-length optimization with three subjects, one per row. The posterior converges to similar but dist... — From Figure 5: Step-length optimization with three subjects, one per row. The posterior converges to similar but distinct preferred gaits per subject, the blind ranking of 3 gaits after 20 trials is shown in column 5, and trials (right) cluster where posterior preference is high.

Abstract

This paper presents a personalized gait optimization framework for lower-body exoskeletons. Rather than optimizing numerical objectives such as the mechanical cost of transport, our approach directly learns from user preferences, e.g., for comfort. Building upon work in preference-based interactive learning, we present the CoSpar algorithm. CoSpar prompts the user to give pairwise preferences between trials and suggest improvements; as exoskeleton walking is a non-intuitive behavior, users can provide preferences more easily and reliably than numerical feedback. We show that CoSpar performs competitively in simulation and demonstrate a prototype implementation of CoSpar on a lower-body exoskeleton to optimize human walking trajectory features. In the experiments, CoSpar consistently found user-preferred parameters of the exoskeleton's walking gait, which suggests that it is a promising starting point for adapting and personalizing exoskeletons (or other assistive devices) to individual users.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong