Assistive robotics and haptics
Assistive arms, prostheses, rehabilitation robots and the haptic feedback that connects them to people.
Start here: seminal papers
Cutaneous Force Feedback as a Sensory Subtraction Technique in HapticsSeminal
August 2011
Replace a haptic device's force feedback with fingertip pressure from wearable cable-driven devices ("sensory subtraction") in simulated needle insertion. With 16 users it overshoots the target less than a visual force display, and under a 50 ms delay it stays stable while full haptic feedback oscillates.
Brain-Computer Interface Controlled Robotic Gait OrthosisSeminal
August 2012
An EEG brain-computer interface switches a treadmill robotic gait orthosis between standing and walking when the user imagines walking. One able-bodied subject and one with paraplegia from spinal cord injury both controlled it on the first try, with 0.812 average correlation to the cues and no missed walk commands.
Autonomy Infused Teleoperation with Application to BCI ManipulationSeminal
March 2015
A shared-control system recognizes objects, infers which grasp the user is reaching for and blends the user's command with an autonomous one, so two people with intracortical brain-computer interfaces can control a 7-DoF arm. On adapted Action Research Arm Test tasks, the subjects completed none of their 15 trials each under direct control, versus 13/15 and 8/15 with assistance.
Human-Robot Mutual Adaptation in Shared Autonomy★ Must read
January 2017
A joystick-driven assistive arm infers both the user's goal and how willing they are to adapt, then nudges adaptable users toward a better goal and gives in to stubborn ones. In a 51-person study, 75% of users in this condition switched to the better goal at least once, and they trusted the robot more than one that always ignored them.
Deep Haptic Model Predictive Control for Robot-Assisted DressingSeminal
September 2017
A PR2 pulls a hospital gown onto a person's arm with model predictive control, using LSTMs trained only in simulation to predict from end-effector force and motion how hard the garment will press on the arm. With a 0.2 s horizon it dressed the full arm in 98.75% of trials with 10 participants and avoided a sleeve catch on the fist in 97.5%.
A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational AutoencoderSeminal
November 2017
An LSTM-based variational autoencoder fuses 17 raw sensor signals from a PR2 feeding robot and flags an anomaly when the current observation is poorly reconstructed relative to a state-dependent threshold. Across 12 types of feeding anomalies it reaches an AUC of 0.8710, above five baseline detectors.
Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer LearningSeminal
January 2018
Pretrain a ConvNet on Myo-armband EMG from 19 people, then give each new user a second network that learns only the difference. This transfer learning improved all three input types tested, reaching 98.31% on 7 gestures over 17 new participants and 68.98% on 18 NinaPro gestures.
Feedback Control of an Exoskeleton for Paraplegics: Toward Robustly Stable Hands-free Dynamic WalkingSeminal
February 2018
The authors port hybrid-zero-dynamics control from bipedal robots to the 12-actuator ATALANTE exoskeleton so complete paraplegics walk without crutches. Three patients walked about 9-10.5 m hands-free at 0.11-0.15 m/s, and a newer learned controller recovers from 750 N pushes in simulation where a fixed gait fails above 300 N.
Towards Robotic Feeding: Role of Haptics in Fork-based Food ManipulationSeminal
April 2018
They recorded 3304 trials of 12 people feeding a mannequin with a force/torque-sensing fork, built a taxonomy of feeding strategies, and classified food into four compliance categories from haptic and motion signals, with a TCN reaching 80.47% accuracy. A robot using one fixed vertical skewering motion acquired bites less reliably than people.
Gaze-based, Context-aware Robotic System for Assisted Reaching and GraspingSeminal
September 2018
The user looks at an object and a UR10 arm plus a robotic glove reach for it, grasp it and use it, sequenced by a finite state machine built on action grammars. 3D gaze is estimated to within 4.68 cm, and with 5 users every reach succeeded, pick-and-place succeeded 96% of the time and pick-pour-place 76%.
Controlling Assistive Robots with Learned Latent Actions★ Must read
September 2019
They train a state-conditioned VAE on demonstrations so that a 1- or 2-DoF joystick input decodes into a full action for a 7-DoF assistive arm. In an eating study users reached the intended morsel in 44 of 50 trials, more often than with shared autonomy, and in a cooking study they finished faster with less joystick input than with end-effector control.
Preference-Based Learning for Exoskeleton Gait OptimizationSeminal
September 2019
CoSpar picks an exoskeleton's gait from a library of precomputed stable gaits using only pairwise preferences ("better than the last trial?") and suggestions like "longer step". In 20 trials per user on the Atalante exoskeleton, its learned preferences matched the blind rankings of all three subjects.
Assistive Gym: A Physics Simulation Framework for Assistive Robotics★ Must read
October 2019
An open-source PyBullet framework with six simulated caregiving tasks, a human model with impairments and preferences, and PPO baselines for four commercial robots. Training the simulated person as an active partner raised the best robot's task success by 30.4 points on average.
Shared Autonomy with Learned Latent ActionsSeminal
May 2020
A joystick-to-arm decoder conditioned on the robot's belief over goals is combined with shared autonomy, so the same low-DoF inputs first steer toward a goal and then control fine manipulation there. In a 10-person eating study with a 7-DoF arm, this combination had the lowest total and idle time on both tasks (p<.05).
Towards Expedited Impedance Tuning of a Robotic Prosthesis for Personalized Gait Assistance by Reinforcement Learning ControlSeminal
June 2020
PICE tunes the 12 impedance parameters of a powered knee prosthesis with policy iteration over a positive-semidefinite quadratic value function, trained offline on logged data and refined online with the user walking. Starting from an offline pre-trained policy cut tuning by 58 impedance updates (about 7 minutes of walking) on average compared with a random start.
Robotic Guide Dog: Leading a Human with Leash-Guided Hybrid Physical InteractionSeminal
March 2021
A Mini Cheetah quadruped guides a blindfolded person on a leash, and its planner lets the leash go slack so the pair can fit through spaces a taut 1.6 m robot-human system cannot. Three adults were guided through doorways and a corridor as narrow as 1.0 m without any collision.
Multimodal Fusion of EMG and Vision for Human Grasp Intent Inference in Prosthetic Hand ControlSeminal
April 2021
Predict which of 13 grasp types a prosthetic-hand user intends by multiplying the probabilities of a forearm-EMG classifier and a gaze-guided camera detector. During reaching, fusion is 95.3% accurate, against 81.64% for EMG and 80.5% for vision alone.
The Design of Stretch: A Compact, Lightweight Mobile Manipulator for Indoor Human EnvironmentsSeminal
September 2021
The design behind the Stretch RE1, a home mobile manipulator that moves the end of its arm in x, y and z with just four actuators (a differential-drive base, a lift and a telescoping arm). It is 34 cm wide and weighs 23 kg. A static-stability model predicts its worst-case payload at full reach as 3.47 kg, against 3.46 kg measured.
MyoSuite -- A contact-rich simulation suite for musculoskeletal motor controlSeminal
May 2022
MyoSuite converts validated OpenSim models of the finger, elbow and hand into MuJoCo and builds 204 contact-rich RL tasks on them, from posing to key turning, pen twirling and Baoding balls, with sarcopenia, fatigue, tendon-transfer and exoskeleton variants. The converted models match OpenSim muscle forces within 4.1% RMS error while simulating 60x to 4000x faster.
Feel the Bite: Robot-Assisted Inside-Mouth Bite Transfer using Robust Mouth Perception and Physical Interaction-Aware ControlSeminal
March 2024
A feeding robot that places food inside the mouth. It tracks the mouth through utensil occlusion with two wrist cameras and picks a compliant controller from a visual-plus-haptic classification of each contact. The system fed 13 care recipients with severe mobility limitations; 9 of them could not lean forward for a standard outside-mouth bite.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.