DexFruit: Dexterous Manipulation and Gaussian Splatting Inspection of Fruit

Check out our website: DexFruit! Arxiv here

A diagram of our method.

Abstract:

Gently handling fragile fruit is a complex task requiring a delicate balance between grip strength and object integrity. We introduce DexFruit, a comprehensive framework for autonomous fruit manipulation using optical tactile sensing. Our approach combines tactile-informed diffusion policies for gentle grasping with FruitSplat, a novel technique for high-resolution 3D damage assessment using Gaussian Splatting. We demonstrate that using optical tactile sensing, autonomous manipulation of fruit with minimal damage can be achieved. Our tactile-informed diffusion policies outperform baselines in both reduced bruising and pick-and-place success rate across three fruits: strawberries, tomatoes, and blackberries. DexFruit demonstrates an average 92% grasping policy success rate with up to a 20% reduction in visual bruising. FruitSplat provides detailed 3D reconstructions for precise damage quantification, advancing the field of agricultural robotics by enabling gentle, damage-minimizing fruit handling.

Authors: Aiden Swann, Alex Qiu, Matthew Strong, Angelina Zhang, Samuel Morstein, Kai Rayle, Monroe Kennedy III (*Both authors contributed equally)

Note: Aiden Swann is supported by NSF GRFP Fellowship No. DGE-2146755. This work was also supported in part by NSF Grant No. 2142773 and 2220867.