Cooperative Mobile Manipulation using Learned Dynamical Systems

TL;DR CALDS lets two mobile manipulators run skills learned separately from a few demonstrations side by side, by choosing a compatible pair of skills and retiming them to avoid whole-body collisions. In simulation it succeeds in 91-100% of runs across three cooperative tasks, where the uncoordinated skills succeed in 0-58%.
What they do
Each of two mobile manipulators has a few skills learned from demonstrations as stable dynamical systems (DSs), learned for one robot alone. CALDS makes them work together in a shared space: it picks a compatible pair of skills, then changes only how fast each robot moves along its learned path, so the robots avoid whole-body collisions and keep task order while still following the demonstrated motion.
How it works
- Learned skills. Each task option is a DS over end-effector pose in SE(3), learned from five teleoperated demonstrations with CDSM (a curve fit to the demonstration) or PUMA (a neural network). Null-space IK maps it to the 9-DoF robot (Clearpath Dingo base, Kinova Gen3 Lite arm).
- Cooperative Policy Selection. Forward-simulate every skill pair, drop pairs whose goals collide, and rank the rest by closeness to the demonstrations and trajectory cost.
- Temporal Adaptation. Optimize a per-step speed scale for each robot over a 30 s horizon with L-BFGS in JAX, penalizing contact between the 8 capsules approximating each robot and task-order violations. Only speed changes, so the learned path and its convergence are kept; on failure the next-ranked pair is tried.
- Static obstacles such as tables are avoided by bending each DS around a fitted ellipse.
Results
- Over 200 randomized simulation runs per scenario, CALDS succeeds in 98-100% (Drawer), 96-98% (Dressing) and 91-96% (Restaurant). The uncoordinated skills reach 54-58%, 28-46% and 0%.
- A whole-body CBF safety filter on the same skills drops to 2% and 36% in Restaurant. ONCol-DMP keeps end-effectors apart in 98.5% of runs but not whole bodies (0-44% success). CALDS deviates less from the learned paths than the CBF filter.
- Planning takes 0.76 s (CDSM) or 1.49 s (PUMA); one action query takes 0.18 ms.
- With state noise and 1 Hz replanning, CALDS-CDSM keeps 98% in Drawer and Dressing but drops to 79% in Restaurant; CALDS-PUMA drops to 82-87%.
Limitations
Simulation only, with simplified tasks: dressing moves a cylinder over the arm, and the drawer is tied to the robot by a string instead of grasped. Planning is centralized for two robots and takes about a second, so the authors note a faster low-level planner may be needed in dynamic scenes. Code is promised on publication.
Videos
Figures



Abstract
Cooperative mobile manipulation requires robots to coordinate their motions while ensuring whole-body collision avoidance and preserving task-relevant behaviors. We propose CALDS (Cooperative Adaptation of Learned Dynamical Systems), a framework that enables mobile manipulators to adapt learned single-agent behaviors for coordinated execution. Building on individual dynamical systems (DSs) learned from demonstrations, CALDS enables cooperative adaptation through two key components: Cooperative Policy Selection, which identifies compatible combinations of learned policies, and Temporal Adaptation, which ensures collision-free cooperation while preserving the spatial structure and convergence properties of the original behaviors. We evaluate CALDS across three diverse scenarios: assistive dressing, restaurant cleaning and serving, and drawer opening and object placement. Quantitative results demonstrate its effectiveness in improving task success and enabling safe, coordinated simultaneous execution while maintaining fidelity to demonstrated behaviors.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
