TidyBot++: An Open-Source Holonomic Mobile Manipulator for Robot Learning

Jimmy Wu, William Chong, Robert Holmberg, Aaditya Prasad, Yihuai Gao, Oussama Khatib, Shuran Song, Szymon Rusinkiewicz, Jeannette Bohg

Seminal December 2024

Learned policy loading a plate into the dishwasher.

TL;DR An open-source holonomic mobile base built from powered casters for $5-6k, carrying a Kinova Gen3 arm and teleoperated with a phone. Diffusion policies trained on 50-100 demonstrations reach 6/10 to 10/10 success on six household tasks, and the holonomic base beats a differential-drive mode 9/10 to 4/10 on countertop wiping.

What they do

Mobile manipulation data is scarce partly because suitable research hardware is expensive, large, or kinematically constrained. TidyBot++ is an open-source mobile manipulator whose base is holonomic: it can move in x, y and rotation independently at any time, like a motorized office chair. The authors pair it with a phone teleoperation interface and show that data collected this way trains working household policies.

How it works

Powered-caster base. Four swerve modules from the FIRST Robotics Competition ecosystem are modified with two 3D-printed wheel mounts and one machined shaft to add a caster offset, which is what makes the base holonomic instead of only omnidirectional. An aluminum T-slot frame carries the modules, a lead-acid battery for the motors, a 768 Wh portable power station, a mini PC and a Kinova Gen3 7-DoF arm.
Control. The low-level controller follows the powered-caster vehicle kinematics of Holmberg and Khatib, extended to a two-dimensional caster offset. The base is commanded in position mode using odometry feedback.
Phone teleoperation. The phone’s 6-DoF pose is streamed through the WebXR API and mapped to base or arm motion; it works on Android and iOS.
Learning. A diffusion policy is trained per task for 500 epochs.

Results

Hardware: $5-6k, 54 x 50 cm footprint, 34 kg, 60 kg conservative payload estimate, 8 hours of teleoperation runtime, assembled in 1 to 2 days. Odometry drifts less than 1 cm per meter and less than 1 degree per full rotation.
Policies from 50 demonstrations (100 for open fridge), 10 rollouts each: open fridge 10/10, take out trash 10/10, wipe countertop 9/10, load dishwasher 7/10, load laundry 7/10, water plant 6/10.
Wiping the countertop in differential-drive mode: demonstrations average 4.03 m and 65.2 s against 2.03 m and 27.4 s holonomic, and the trained policy succeeds 4/10 against 9/10.

Limitations

The base does not backdrive well, because of high steering friction from the 12.8 steer gear ratio and the small 14 mm caster offset. Fixing this would need more custom machined parts, which would make the open design less accessible.

Videos

Learned policy moving laundry from a basket into the washing machine.

Learned policy picking up a cup at the sink to water a plant.

Figures

From Figure 1: We develop an open-source mobile manipulator with a holonomic base (left), and show that it can perfor... — From Figure 1: We develop an open-source mobile manipulator with a holonomic base (left), and show that it can perform a variety of household tasks in a real apartment home (right).

From Figure 3: Our mobile base is designed to be modular and easily reconfigurable. It has very few components and ca... — From Figure 3: Our mobile base is designed to be modular and easily reconfigurable. It has very few components and can be assembled in 1 to 2 days.

From Figure 4: Isometric and top views of a simplified caster, showing the caster offsets bx and by, wheel radius r, ... — From Figure 4: Isometric and top views of a simplified caster, showing the caster offsets bx and by, wheel radius r, steer and roll joints, and caster module placement from the base origin.

From Figure 5: In the wipe countertop task, the differential drive robot is forced to take a less efficient path as i... — From Figure 5: In the wipe countertop task, the differential drive robot is forced to take a less efficient path as it is subject to nonholonomic constraints.

Abstract

Exploiting the promise of recent advances in imitation learning for mobile manipulation will require the collection of large numbers of human-guided demonstrations. This paper proposes an open-source design for an inexpensive, robust, and flexible mobile manipulator that can support arbitrary arms, enabling a wide range of real-world household mobile manipulation tasks. Crucially, our design uses powered casters to enable the mobile base to be fully holonomic, able to control all planar degrees of freedom independently and simultaneously. This feature makes the base more maneuverable and simplifies many mobile manipulation tasks, eliminating the kinematic constraints that create complex and time-consuming motions in nonholonomic bases. We equip our robot with an intuitive mobile phone teleoperation interface to enable easy data acquisition for imitation learning. In our experiments, we use this interface to collect data and show that the resulting learned policies can successfully perform a variety of common household mobile manipulation tasks.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong