Matt Strong

Matt Strong

CS PhD at Stanford University

← Paper Digest

Human Preference-Based Learning for High-dimensional Optimization of Exoskeleton Walking Gaits

Maegan Tucker, Myra Cheng, Ellen Novoseller, Richard Cheng, Yisong Yue, Joel W. Burdick, Aaron D. Ames

Seminal March 2020

TL;DR LineCoSpar learns a person's preferred exoskeleton walking gait from pairwise comparisons by searching one random line of the gait parameter space at a time. On the Atalante exoskeleton it tuned six gait parameters for six able-bodied users in 30 trials each.

What they do

Exoskeleton gaits should be comfortable, but no numeric metric is known to track comfort, so the authors learn from which of two gaits a user prefers. Earlier preference-learning methods handled at most two gait parameters. LineCoSpar scales this to many parameters and is run on the Atalante lower-body exoskeleton over six gait parameters with six able-bodied subjects. The collected preferences are then analyzed to find what makes a gait preferred.

How it works

  • Preference model. A Gaussian process posterior over gait utilities is fit to pairwise preferences with a Laplace approximation, using a sigmoid link function that the authors found works better than the Gaussian one of prior work.
  • One line at a time. Each iteration draws a random line through the current best gait and updates the posterior only over points on that line plus previously tested gaits. The cost is constant in the number of dimensions, where CoSpar’s grows exponentially.
  • Thompson sampling. A utility function is sampled from the posterior and its maximizer is executed; the user compares it with the previous gait or reports no preference.
  • Coactive feedback. The user may also suggest a change along one parameter, which is stored as a preferred gait.

Results

  • Simulation: on the Hartmann3 benchmark LineCoSpar reaches higher objective values faster than CoSpar and stays robust under noisy preferences. On random 6-dimensional polynomials it samples high-valued actions within about 20 iterations with coactive feedback.
  • Human trials: after 30 trials, the predicted best gait was checked against four random gaits. It won all four comparisons for four of six subjects, three for one subject and one for the other.
  • A Linear Inverted Pendulum cost and its dynamic-stability term capture the preferences of five of six subjects; a static-stability cost fully predicts those of the remaining subject.

Limitations

All subjects were able-bodied; subjects with paraplegia likely have different preferences and are left to future work. The method does not account for preferences changing over time. Gaits come from a pre-computed library, and LineCoSpar is not compared against other optimizers because no other preference-based Gaussian process method is tractable in high dimensions.

Figures

From Figure 1: The Atalante exoskeleton, which has 12 actuated joints, and the six gait parameters explored in the ex...
From Figure 1: The Atalante exoskeleton, which has 12 actuated joints, and the six gait parameters explored in the experiments: step length, step duration, step width, maximum step height, pelvis roll and pelvis pitch.
From Figure 6: Experimental procedure. Subjects gave a preference between each consecutive pair of gaits, plus coacti...
From Figure 6: Experimental procedure. Subjects gave a preference between each consecutive pair of gaits, plus coactive feedback, over 30 gait trials. They then unknowingly entered validation, comparing the posterior-maximizing gait against four randomly selected gaits.
From Figure 3: Mean objective value of the sampled actions on the Hartmann3 (H3) and Hartmann6 (H6) benchmarks, avera...
From Figure 3: Mean objective value of the sampled actions on the Hartmann3 (H3) and Hartmann6 (H6) benchmarks, averaged over 100 runs. LineCoSpar converges to higher values faster than CoSpar, which is intractable on the 6-dimensional space.
From Figure 8: A single exoskeleton step with the Linear Inverted Pendulum Model overlaid.
From Figure 8: A single exoskeleton step with the Linear Inverted Pendulum Model overlaid.
Abstract

Optimizing lower-body exoskeleton walking gaits for user comfort requires understanding users' preferences over a high-dimensional gait parameter space. However, existing preference-based learning methods have only explored low-dimensional domains due to computational limitations. To learn user preferences in high dimensions, this work presents LineCoSpar, a human-in-the-loop preference-based framework that enables optimization over many parameters by iteratively exploring one-dimensional subspaces. Additionally, this work identifies gait attributes that characterize broader preferences across users. In simulations and human trials, we empirically verify that LineCoSpar is a sample-efficient approach for high-dimensional preference optimization. Our analysis of the experimental data reveals a correspondence between human preferences and objective measures of dynamicity, while also highlighting differences in the utility functions underlying individual users' gait preferences. This result has implications for exoskeleton gait synthesis, an active field with applications to clinical use and patient rehabilitation.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.