HALO: Human-Aligned Data Collection for WhoLe-body MObile Manipulation

Shuai Yang, Fangyi Xu, Yishu Shen, Kecheng Zheng, Yinghao Xu

November 2026

The learned policy runs the long-horizon box-to-conveyor task on the mobile base, 3x speed.

TL;DR HALO collects whole-body mobile manipulation demonstrations by having the operator move the robot's arms by hand while walking a passive, base-free platform, then inpaints the operator's hands out of the video. Collection is up to 2.9x faster than Mobile ALOHA, and a fine-tuned pi0.5 policy completes a six-stage shelf-to-conveyor task in 11 of 15 trials.

What they do

VR teleoperation gives indirect control without force feedback, and Mobile ALOHA’s leader arms keep the operator away from the grippers while the powered base gets in the way. HALO has the operator move the robot arms by hand (kinesthetic teaching) while walking a light, base-free platform, then processes the recordings to match autonomous deployment.

How it works

Manipulation. A redesigned end-effector, less than half the original size with about 40° more pitch range, gives direct visual and force feedback.
Mobility. During collection the differential-drive base is replaced by passive wheels plus two fixed-direction wheels on the drive axis, keeping the same nonholonomic constraint. Pushing it takes 2.3 N, vs 13.0 N for Mobile ALOHA.
Deployment alignment. An affine map and an estimated response delay turn encoder-measured platform motion into base velocity commands; operator hands are segmented (SAM2) and inpainted out of the video.
Policy. pi0.5 is fine-tuned per task to output arm joints and base velocity together, run with asynchronous real-time action chunking.

Results

With one operator, the same tasks and a one-hour budget, demonstrations are 2.9x (Push Chair) and 2.8x (Disassemble Lego) faster than with Mobile ALOHA, and Lego demonstration success is 20 points higher.
Policies succeed 30/40 on Disassemble Lego and 11/15 on the six-stage box task. Policies trained on VR or Mobile ALOHA data from the same time budget fail both tasks.
Inpainting the hands out raises Push Chair success from 55% to 85%.

Limitations

The paper has no limitations section. The collection comparison uses a single operator, and policies are trained per task. Each dataset needs a hand-inpainting pass and a calibration of the passive platform against the powered base. Active perception (moving the camera viewpoint) is left to future work.

Videos

An operator collects a demonstration by guiding the arms by hand while walking the passive platform.

Figures

From Figure 1: Mobile ALOHA leaves a perception gap and a mobility gap between operator and robot; HALO puts the oper... — From Figure 1: Mobile ALOHA leaves a perception gap and a mobility gap between operator and robot; HALO puts the operator at the end-effectors and lets them walk naturally.

From Figure 2: (a) Data collection by bimanual kinesthetic teaching while walking. (b) Learned policies doing precise... — From Figure 2: (a) Data collection by bimanual kinesthetic teaching while walking. (b) Learned policies doing precise bimanual manipulation, coordinated base and arm motion, long-horizon tasks and navigation.

From Figure 8: Top, demonstration time and success rate for HALO, Mobile ALOHA and VR on Push Chair and Disassemble L... — From Figure 8: Top, demonstration time and success rate for HALO, Mobile ALOHA and VR on Push Chair and Disassemble Lego. Bottom, typical failures: VR lacks precise gripper control, and the mobile base constrains operator movement.

Abstract

Learning whole-body mobile manipulation requires demonstrations that coordinate bimanual actions with locomotion across extended workspaces. However, existing interfaces trade off direct robot control, perception and action alignment, and natural operator mobility, making such demonstrations difficult to collect. We introduce HALO, a human-aligned data collection system for whole-body mobile manipulation. The system aligns the operator's perception and action space with the robot while preserving direct physical feedback and natural locomotion. Operators guide the robot end-effectors through kinesthetic teaching and move a low-resistance platform without the deployed mobile base while performing bimanual manipulation. This interface enables more natural whole-body demonstrations, but the close human-robot coupling introduces discrepancies between the demonstration and deployment settings. We address these discrepancies with deployment-aligned demonstration processing, which calibrates and temporally aligns platform motion with the deployed base response and masks and inpaints demonstrator hands in recorded observations. Compared with Mobile ALOHA and VR teleoperation, HALO enables up to 2.9x faster demonstration collection while maintaining comparable or higher success rates. Policies trained with HALO perform precise manipulation tasks, including zip-tie fastening, and achieve a 73.3% success rate on a six-stage long-horizon mobile manipulation task involving repeated coordinated base and arm motion. Project page: https://robbyant-research.github.io/Lingbot-ALOHA/

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong