SPICE: Stochastic Policy for Interaction in Cluttered Environments

TL;DR SPICE is a diffusion policy that reaches targets inside occluded plant canopies using only tactile arrays on the gripper, learning to push through foliage, move around stems and push blocking branches down. Over 180 real-robot trials it manoeuvred around branches in 19 of 20 trials versus 8 of 20 for the best alternative tactile input, and reached targets in 35 of 40 zero-shot trials on real lime and tomato plants.
What they do
Reaching fruit or a pruning site inside dense foliage means touching leaves and branches while the camera view is blocked. SPICE is a vision-free diffusion policy for an xArm6 that reaches a target using only tactile feedback from its gripper. One policy, trained on 240 teleoperated demonstrations without behaviour labels, learns three responses: pushing through compliant foliage, moving around a vertical branch, and pushing down a horizontal branch that blocks the path.
How it works
- Sensing. Two Xela uSkin arrays of 4x4 three-axis taxels on the gripper front form one tactile tensor; a small CNN keeps the taxels’ spatial layout.
- Contact signal. Contact appears in only about 30% of the demonstrations, so an explicit contact-intensity value, zero in free space, joins the conditioning.
- Policy. A Diffusion Policy U-Net, conditioned on the tactile embedding, contact signal and current and target joint configurations, predicts 16 joint positions, executes 8 and replans.
- Data. GELLO teleoperation on a mock canopy of balsa branches and artificial leaves, with the operator watching a tactile heat map.
Results
- Branch manoeuvring: 19/20 for SPICE against 8/20 for the best alternative tactile representation; without the contact signal SPICE fails all 10 trials.
- Against the reactive RICE controller, both manoeuvre 30/30, but only SPICE meets the branch-manipulation criterion (20/20). RICE reaches more accurately (1.17 vs 2.78 cm target error).
- Against an image-conditioned Diffusion Policy under occlusion: 19/20 vs 13/20 for plain reaching, 18/20 vs 0/20 for manoeuvring.
- Zero-shot on unseen real plants: 20/20 lime and 15/20 tomato trials; the other five were safety stops.
Limitations
Demonstrations come from a single operator without haptic feedback. Tactile sensing covers only the end-effector faces, which the authors link to the tomato safety stops, and branches that slid beyond it were sometimes released by hand.
Figures




Abstract
Agricultural tasks such as harvesting and pruning require robots to reach through dense foliage, where contact is often unavoidable and vision is obstructed. Intermittent contact creates an imbalance between free-space and contact-rich observations, complicating the learning of contact-responsive behaviours. We present SPICE, a tactileconditioned diffusion policy for reaching targets in dense plant canopies using tactile sensing as the sole environmental feedback. SPICE exhibits three behaviours: moving through compliant foliage, manoeuvring around partially obstructing stems, and manipulating branches that block the target path. To the best of our knowledge, SPICE is the first tactile-conditioned diffusion policy for target reaching through physically cluttered plant canopies. Across over 180 real-robot trials, SPICE was evaluated against tactile-conditioned, image-conditioned, and reactive-control baselines. Spatial tactile information and explicit contact conditioning proved important, with SPICE achieving 95% branch-manoeuvring success versus 45% for the strongest tactile baseline. SPICE also achieved branch manipulation beyond the reactive controller and remained effective under visual occlusion, where the image-conditioned DP failed. Zero-shot transfer across 40 trials on unseen tomato and lime plants achieved 87.5% success rate. These results demonstrate the potential of tactile-conditioned generative policies for interaction in visually occluded plant environments.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
