Adaptive Hybrid Force-Vision Control with EMG-Derived References for Deformable Bottle Manipulation

Jeffrey Lee, Geoffrey Hanks, Evangeline Bouganim, Yue Hu

November 2026

TL;DR A TIAGo Pro gripper squeezes lab wash bottles by tracking grip-force profiles recorded from a person's forearm EMG, combining fingertip force feedback with camera-estimated bottle deformation. Across 12 bottle, volume and squeeze-count conditions it dispenses within 6.54 mL of the human demonstrations on average, against 15.92 mL for force control alone.

What they do

Squeeze bottles are common lab equipment, but force feedback alone says little about how deformed a bottle is: similar forces appear at different deformation levels, and a force-controlled gripper does not let the bottle recover between squeezes. The authors record a person squeezing a bottle with surface EMG on the forearm, map the EMG to a grip-force reference, and have a robot gripper reproduce it with a controller that mixes force and vision feedback.

How it works

EMG to force. EMG from the flexor digitorum superficialis is recorded while the user squeezes a foam block with FSR caps on the thumb and middle finger. A small MLP fitted on 50 trials maps EMG to the FSR reading the robot should feel.
Sensing on the robot. Custom rounded fingers on the TIAGo Pro gripper keep 4x4 FSR pads in contact as the bottle caves in. A wrist-mounted RealSense D405 point cloud gives deformation as the percentage width reduction of the thinnest slice relative to the widest.
Hybrid controller. A position-based law in which an EMG-dependent factor shifts weight between force error and deformation feedback, designed so force dominates while squeezing and deformation dominates when EMG is low and the gripper should reopen. The force gain rises once deformation exceeds 20%.

Results

The vision estimate has 5.26% (small bottle) and 6.96% (large) mean absolute error up to 65% deformation but degrades beyond 70%, so gripper closing was capped there.
Over 24 condition-demonstration pairs with 5 robot trials each, hybrid control’s error against the human-dispensed volume is 6.54 mL MAE, against 15.92 mL for force control, with similar trial-to-trial spread (mean SD 3.14 vs 3.70 mL). For the 40 mL target it is 4.93 vs 25.43 mL; at 80 mL force control has lower error in several conditions.
The hybrid gripper reopens between squeezes as EMG drops; the force-only gripper closes to about 0.75 and stays there, so its volumes cluster at 60-80 mL regardless of target.

Limitations

Demonstrations come from a single user (one of the authors), and the recorded EMG is replayed rather than used live. Only two plastic wash bottles and water were tested, and the deformation estimate fails beyond about 70% squeeze. Other bottle materials, liquid viscosities and glass containers are left to future work.

Figures

From Figure 1: The framework. EMG is mapped to grip force, and the processed EMG signal and the vision-estimated bott... — From Figure 1: The framework. EMG is mapped to grip force, and the processed EMG signal and the vision-estimated bottle deformation feed the adaptive hybrid controller that commands the gripper.

From Figure 5: Experimental setup with the TIAGo Pro, a RealSense D405 camera and an FSR-equipped gripper squeezing a... — From Figure 5: Experimental setup with the TIAGo Pro, a RealSense D405 camera and an FSR-equipped gripper squeezing a plastic bottle to dispense water into a cup on a digital scale.

From Figure 9: Volume dispensed by adaptive hybrid control (left) and force control (right) for each bottle and squee... — From Figure 9: Volume dispensed by adaptive hybrid control (left) and force control (right) for each bottle and squeeze count. Each pair of boxes is five robot trials for human demonstrations 1 and 2; red lines mark the human-dispensed volumes.

Abstract

Robotic automation of laboratory tasks has become a growing interest to remove researchers from hazardous environments and repetitive procedures. However, even equipment as simple as squeeze bottles poses a challenging manipulation problem due to its deformability. This paper presents an adaptive hybrid vision-force controller for robotic grippers to effectively manipulate laboratory squeeze bottles to dispense liquid. This approach integrates force and vision feedback according to the interaction state, enabling effective squeezing while accounting for bottle deformation and recovery. The proposed controller is validated by comparing its performance with a pure force controller on 12 combinations of conditions consisting of 2 target dispense volumes, 2 plastic squeeze bottles, and 1 to 3 squeezes used to dispense. Human demonstrations of these operating conditions were collected using electromyography (EMG) sensing mapped to a desired grip force, which was then used as the force reference for the controllers to squeeze the bottles. Results demonstrated that the proposed adaptive hybrid vision-force controller achieved more accurate liquid dispensing with a mean absolute error (MAE) of 6.54 mL compared to a MAE of 15.92 mL from the pure force controller, while also replicating the human action recorded from the EMG signal more closely.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong