Learning a Speed-adaptive Hip Exoskeleton Control Policy Via Sim-to-real Reinforcement Learning

Bin Li, Zhimin Hou, Jiacheng Hou, Zenian Liang, Tong Wu, Teng Ma, Chenglong Fu

September 2026

TL;DR They learn when a hip exoskeleton should push with sim-to-real RL on a musculoskeletal model walking at varying speeds, then tune only two torque gains on the real device from users' pairwise preferences. In free walking it cut negative mechanical power by 55% compared with a delayed-feedback controller.

What they do

A hip exoskeleton has to get both the timing and the magnitude of its assistance right, at any walking speed. The authors learn the timing in simulation with RL against a musculoskeletal model. On the real device they tune only the magnitude, from users’ pairwise preferences, so the online search has just two parameters.

How it works

Simulated walker. A PPO policy drives a MuJoCo musculoskeletal model (53 DoF, 22 muscle-tendon units). It first imitates reference gait, then drops imitation and tracks commanded speeds of 0.5-1.7 m/s and accelerations.
Exoskeleton teachers. Separate policies for steady walking, acceleration and deceleration first learn to push in the direction the hip is moving. They then co-adapt with the human policy to reduce hip muscle activation.
Distillation. A TCN student uses only hip angles and velocities from two IMUs. It runs on a Raspberry Pi 5 without estimating gait phase.
Preference tuning. Two gains scale extension and flexion torque. A Gaussian-process preference model proposes pairs of settings; the user walks 30 s with each and says which is better.

Results

Simulation: the walker covers 0.5-1.7 m/s even though its reference motion was recorded only at 1.25 m/s. At similar torque to delayed output feedback control (DOFC) and a spline controller, the policy delivers more positive power, and it lowers rectus femoris activation.
Treadmill (6 participants): mean positive power was 5.14 W with DOFC, 5.59 W unpersonalized and 8.38 W personalized. Personalization raised peak torque from 5.13 to 8.33 Nm.
Muscle activity (EMG) did not consistently follow: more positive power did not necessarily mean less muscle activation.
Free walking at self-varied speed (4 participants): compared with DOFC, negative power fell 55% (p=0.030) and the resistance ratio 27%, while RMS torque fell only 9%.

Limitations

Only level-ground walking and only a hip exoskeleton. On hardware, the personalized controller was compared only with DOFC and its own unpersonalized version; the other baselines ran in simulation only.

Figures

From Figure 1: Overview. (a) A human control policy learns to walk at varying speeds. (b) Hip exoskeleton teacher pol... — From Figure 1: Overview. (a) A human control policy learns to walk at varying speeds. (b) Hip exoskeleton teacher policies learn by interacting with the musculoskeletal model. (c) The teachers are distilled into a TCN student that uses onboard sensors. (d) Assistance magnitude is personalized online from user preferences.

From Figure 3: Commanded assistive torque while speeding up from 0.9 to 1.6 m/s and slowing back down, for their poli... — From Figure 3: Commanded assistive torque while speeding up from 0.9 to 1.6 m/s and slowing back down, for their policy and five baselines. MLP-PPO and LSTM-PPO were trained only at 1.25 m/s.

From Figure 4: Negative mechanical power and resistance ratio in free walking, DOFC vs. theirs. Resistance ratio is t... — From Figure 4: Negative mechanical power and resistance ratio in free walking, DOFC vs. theirs. Resistance ratio is the fraction of samples with negative assistive power within 100 ms after the hip angular velocity crosses zero.

Abstract

Providing personalized exoskeleton assistance across varying walking speeds remains challenging. Existing online optimization methods are sample-inefficient, requiring extensive human-in-the-loop (HIL) evaluations to optimize the entire assistive torque profile. Sim-to-real reinforcement learning (RL) offers a promising alternative but cannot directly account for individual user preferences. We propose a framework integrating sim-to-real RL with online preference learning for personalized exoskeleton assistance. Specifically, assistance timing is learned in simulation by training RL policies with human musculoskeletal models across varying walking speeds. The learned policies are then distilled and deployed on a physical hip exoskeleton using onboard sensory observations. Gaussian-process-based preference learning further personalizes the assistance magnitude through pairwise user comparisons. By decoupling assistance timing learning in simulation from magnitude optimization in real-world experiments, our framework substantially reduces the online optimization space. Human-subject experiments demonstrate efficient identification of personalized assistive torque profiles across varying walking speeds with fewer real-world evaluations.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong