AsymEEF: Bimanual Manipulation with Asymmetric End Effectors
TL;DR AsymEEF maps grippers and dexterous hands into one shared latent state, so bimanual demonstrations teleoperated with one pair of end effectors can be replayed on other pairs and filtered by task success. On six real tasks, 88.7% of transferred demonstrations succeed on replay, against 20.8% and 45.2% for two prior transfer methods.
What they do
Bimanual demonstrations are tied to the hands they were recorded with, and N hand types per arm give N² ordered pairs to teleoperate. AsymEEF reuses demonstrations from a few source pairs on other pairs of two-, three- and five-fingered end effectors. It also trains a single visuomotor policy across all the pairs.
How it works
- Shared end-effector state. A topology-biased Transformer turns each hand’s actuators (one token per actuator, with joint type, axis and mounting) plus the wrist pose into a 24-D latent. A decoder conditioned on the target hand’s morphology recovers its actuator values and wrist pose.
- Two kinds of anchors. A kinematic teacher optimizes target hand states to match the grasp center, opening width and approach direction of a source state. X-Calib adds real grasp keyframes taught by hand (216 frames for XHAND–G2, under 1 h) to capture assembly offsets, actuator coupling and contact compliance.
- Replay filtering. Transferred trajectories are run through IK and replayed in simulation or on the real robot. They are kept only if they are feasible, collision-free, make the right contacts and complete the task.
- Policy. ACT or π0.5 is trained on the shared latent action space, so data from different hand pairs can be mixed without per-hand heads.
Results
- Real replay success (xArm7 arms with XHAND hands and G2 grippers) averages 88.7% over six tasks, against 20.8% for LAD and 45.2% for CEI. On CPU Installation it is 79.0% against 31.0% and 0.0%.
- In simulation, progressive spatial expansion accepts 4,959 of 6,549 candidates (75.7%), a mean of 16.6 replay-valid trajectories per source demonstration.
- Policies trained on the generated data reach 83.3% mean success on the six real tasks with both ACT (55-100% per task) and π0.5 (64-100%), with 11 trials per task.
- Removing X-Calib roughly triples the cross-hand error on real hardware data (MAE 0.2088 to 0.6181).
Limitations
Tabletop tasks only, with two identical arms. X-Calib needs calibration keyframes collected by hand, and the method assumes the target hand can physically perform the task. Policy success is reported only for policies trained on AsymEEF’s own data, not against policies trained on the baselines’ data.
Figures




Abstract
Teleoperated demonstrations are costly to collect and rarely reusable across end-effectors with different kinematics and action spaces. This burden grows in bimanual manipulation, where each end-effector pair may require new data. We present AsymEEF, a framework for executable cross-end-effector demonstration generation and unified bimanual policy learning. Given a small source dataset, AsymEEF transfers demonstrations to heterogeneous combinations of two-, three-, and five-fingered end-effectors without additional teleoperation. Kinematic teacher anchors provide geometric supervision, while sparse real-world X-Calib anchors correct discrepancies in grasp execution. A shared encoder maps wrist poses and actuator states to a morphology-agnostic latent space, and a morphology-conditioned decoder reconstructs target commands. Generated trajectories undergo replay filtering for task-level executability. The representation also supports a unified bimanual visuomotor policy trained across multiple pairs. Across four simulated and six real-world tasks, AsymEEF improves replay success over adapted transfer baselines, achieves a mean replay-valid expansion gain of 16.6 times in simulation, and is evaluated through downstream policy training on real robots. Project website: https://asymeef.vercel.app/.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
