HumanoidExo: Scalable Whole-Body Humanoid Manipulation via Wearable Exoskeleton
- arXiv
- Ask alphaXiv
- Project page
- Humanoids and mobile manipulation
- Imitation learning and data collection
TL;DR A wearable arm exoskeleton with a back-mounted LiDAR records a person's whole-body motion and retargets it to a Unitree G1 humanoid. Adding 195 exoskeleton demos to 5 teleoperated ones lifts tabletop pick-and-place success from 5% to about 80%, and the robot learns to walk to the table from exoskeleton data alone.
What they do
Teleoperating a humanoid needs the robot and a skilled operator for every demo, while human video and simulation carry an embodiment gap. HumanoidExo records a person wearing an arm exoskeleton and a back-mounted LiDAR, and retargets the motion to a Unitree G1 with two Inspire hands. The data trains HE-VLA, a vision-language-action policy paired with an RL balance controller.
How it works
- Upper body in joint space. The exoskeleton reads all seven arm joints. Its forward kinematics gives the upper-arm pose, which IK maps onto the robot’s first three arm joints; the elbow angle is copied directly; two IMUs (forearm and glove) give the wrist rotation for the last three joints.
- Lower body as commands. FAST-LIO LiDAR odometry records the operator’s displacement, heading and height, which become forward-velocity, yaw-rate and torso-height commands.
- Policy. DexVLA, pretrained without G1 data, is fine-tuned on the data. Fine-tuned alone it often made the robot fall, so an actor-critic RL controller, trained with a joint-range curriculum, mixed walking/squatting environments and left-right symmetry losses, keeps balance while tracking the commands.
Results
- PlaceToy: 5 teleoperated demos alone give 5% success; adding 195 exoskeleton demos gives about 80%, comparable to 200 teleoperated demos. Mixed data also beats teleop-only on four new objects (present in the exoskeleton data) and in a new environment.
- Walk & PlaceToy: the 5 teleoperated demos contain no walking, yet with 195 exoskeleton demos the walking phase succeeded in every trial. Dragged away from the table, the robot walked back, even from beyond distances seen in training.
- PlaceLaundry (squat, load a washer, stand): 80% / 75% success on seen / unseen clothes with mixed data, versus 80% / 80% with 200 teleoperated demos and 5% / 5% with 5.
Limitations
The paper has no limitations section. All policies still use 5 teleoperated demos, and operators were recruited with upper-body dimensions close to the G1’s. The robot’s head is fixed while the operator’s moves. Experiments cover one robot and three tasks, and the only baselines are teleoperated-data budgets.
Videos
Figures
Abstract
A significant bottleneck in humanoid policy learning is the acquisition of large-scale, diverse datasets, as collecting reliable real-world data remains both difficult and cost-prohibitive. To address this limitation, we introduce HumanoidExo, a novel system that transfers human motion to whole-body humanoid data. HumanoidExo offers a high-efficiency solution that minimizes the embodiment gap between the human demonstrator and the robot, thereby tackling the scarcity of whole-body humanoid data. By facilitating the collection of more voluminous and diverse datasets, our approach significantly enhances the performance of humanoid robots in dynamic, real-world scenarios. We evaluated our method across three challenging real-world tasks: table-top manipulation, manipulation integrated with stand-squat motions, and whole-body manipulation. Our results empirically demonstrate that HumanoidExo is a crucial addition to real-robot data, as it enables the humanoid policy to generalize to novel environments, learn complex whole-body control from only five real-robot demonstrations, and even acquire new skills (i.e., walking) solely from HumanoidExo data.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.