AffectiveVLA: Safe and Intent-Aware VLA Control for Human-Humanoid Social Interaction
TL;DR A fast video classifier recognizes whether a person is starting a handshake, fist bump or hug and prompts a VLA on a Unitree G1 humanoid, which stays still until the intent is confident; per-interaction arm compliance and a spoken stop guard the contact. Intent gating raises GR00T's on-robot macro-F1 from 0.75 to 0.90.
What they do
AffectiveVLA lets a Unitree G1 humanoid with BrainCo dexterous hands shake hands, fist-bump and hug. A small video classifier recognizes which interaction the person is starting and turns it into the VLA’s language prompt; per-interaction arm compliance and a voice-triggered stop supervise the contact. The authors also compare four VLAs finetuned on the same data.
How it works
- Data. 1,270 VR-teleoperated episodes from 19 participants (handshake, fist bump, hug, no interaction); the 288 test episodes come from 4 unseen participants.
- Intent gate. A TinyViT + VideoLightFormer classifier reads about one second of two onboard RGB views. An intent is committed only after its probability exceeds 0.6 for three consecutive steps; until then the robot does not move.
- Execution. The VLA predicts whole-body motion tokens for the SONIC controller plus finger commands at 5 Hz, executed at 50 Hz. Fingers stop closing once contact is detected.
- Safety. Each interaction sets its own impedance scaling for the 14 arm joints. Saying “stop” or “hold on” latches a stop that softens and lowers the arms and opens the hands.
Results
- On the robot (5 participants, 150 trials per policy), GR00T is the best standalone VLA at macro-F1 0.75. Being-H0.5 scores 0.12 despite offline action error within 2% of the best.
- With the intent gate, GR00T reaches 0.90: fist bumps rise from 0.50 to 0.78 and hugs reach 1.00.
- The intent model runs at 6.27 ms p95, commits 77% of intents before contact (median 2.1 s ahead) and made no false activations on 54 no-interaction validation episodes. Both stop phrases were detected in 10/10 trials.
Limitations
Few participants, three interaction types, one robot and one whole-body controller. False activations during long continuous operation are untested, and the intent model may generalize poorly to new people, especially for fist bumps. Lower stiffness gives no formal bound on contact force, and the compliance user study is still ongoing.
Figures
Abstract
Social touch interactions, such as handshakes, fist bumps, and hugs, require humanoid robots to infer human intent, generate contact-rich motions, and maintain safe physical contact. We present AffectiveVLA, an intent-gated Vision-Language-Action (VLA) pipeline deployed on a Unitree G1 humanoid with dexterous hands. A lightweight RGB-based model recognizes interaction intent from two onboard cameras and converts it into a language prompt for the VLA policy. Low-confidence predictions suppress motion execution, reducing unintended activations. During physical contact, interaction-specific compliance profiles adjust arm impedance, while an independent voice-triggered mechanism initiates a latched safe-stop procedure. Using VR teleoperation, we collected approximately 1,300 multimodal interaction episodes and fine-tuned four VLA models: GR00T, RLDX-1, Being-H0.5, and UnifoLM-VLA-0. On-robot evaluation identifies GR00T as the best-performing standalone model. Integrating intent recognition increases its macro-F1 score from 0.75 to 0.90, while the intent model achieves a p95 inference latency of 6.27 ms. These results demonstrate that explicit intent gating improves the reliability of VLA-based social interaction without compromising real-time operation.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.