From Prediction to Action: Learning Physical Context for Deformable Linear Object Manipulation

Chengxiao Dong, Alessio Caporali, Hongyu Lan, Gianluca Palli

November 2026

TL;DR For shaping cables and ropes, knowing the stiffness and friction helps predict the next shape but not choose the action; a latent of recent interaction history does. Trained only in simulation, the history-conditioned policy brings real ropes within 10 mm of the target shape in 88.3% of rollouts on a UR5, against 48.3% for a diffusion-policy baseline.

What they do

Cables and ropes with the same shape respond differently to the same action, depending on stiffness, friction and interaction history. The paper asks whether the information that best predicts a deformable linear object’s (DLO’s) response is also the most useful for choosing actions. In planar shape control with one grasp per action, it compares three kinds of physical context (ground-truth parameters, estimated parameters, and a learned latent of recent interactions) for both forward prediction and executed control.

How it works

GPF policy. Geometry-Pointer-Flow encodes the current and target centerlines (51 points) with a Transformer. A pointer picks one of 24 grasp segments, and a rectified-flow model generates the planar translation and rotation for that grasp.
Physical context. Oracle gives Young’s modulus and friction from the simulator. Estimated infers both from up to five past interactions with a frozen estimator. Hist encodes the same history into a 32-dimensional latent trained jointly with the policy, injected by concatenation or FiLM.
Predictive utility. Separate forward models predict the next shape from the executed action, with each context.
Data. 90,106 episodes in AGX Dynamics with randomized stiffness and friction. Policies are transferred zero-shot to a UR5 with a wrist camera.

Results

Geometry-only GPF has 32.8% lower one-step shape error than a DLO diffusion policy (9.257 vs 13.782 mm), with 18.7x fewer parameters and 4.41 ms instead of 851 ms inference.
Estimated parameters predict slightly better than Hist in simulation but give no detectable control gain; Hist-FiLM lowers executed error by 9.9% over geometry-only GPF across 470 paired trials.
Closed loop in simulation, 10 actions: Hist-FiLM reaches 5 mm in 36% of rollouts, against 25% for geometry-only GPF and 20% for the diffusion policy.
Real robot, three ropes on two surfaces, 60 rollouts per controller: Hist-FiLM reaches 10 mm in 88.3% of rollouts, geometry-only GPF in 76.7%, the diffusion policy in 48.3%.

Limitations

The task is planar, with a single grasp per action and one-step reachable targets in training. The experiments do not isolate why history helps, nor what the latent encodes beyond stiffness and friction; the weaker hardware result of the estimator may be sim-to-real degradation. Real-robot trials cannot replay the same initial state, so they are paired only by target and condition.

Figures

From Figure 1: Overview of the predictive and decision utility of physical context. (a) Controlled simulation generat... — From Figure 1: Overview of the predictive and decision utility of physical context. (a) Controlled simulation generates matched state-action pairs while varying stiffness and friction. (b) Three alternative physical contexts: Oracle uses the simulator's ground-truth parameters; Estimated infers them from the interaction history with a pretrained, frozen parameter estimator; Hist encodes the same history into a latent with an encoder trained jointly with the downstream model. (c) The same context conditions either a forward predictor, whose next-shape error defines predictive utility, or a goal-conditioned GPF policy, whose executed shape error defines decision utility. (d) We test whether predictive utility aligns with decision utility. (e) Deployable context representations are evaluated zero-shot on the real robot.

From Figure 3: Geometry-Pointer-Flow policy with interaction-history conditioning. GPF selects where to interact thro... — From Figure 3: Geometry-Pointer-Flow policy with interaction-history conditioning. GPF selects where to interact through a query–key pointer and generates how to move the selected contact through rectified flow. The latent context modulates segment features before grasp selection (1) and the post-selection flow condition (2).

From Figure 5: Representative zero-shot real-robot manipulation processes of GPF Hist-FiLM under three DLO-surface co... — From Figure 5: Representative zero-shot real-robot manipulation processes of GPF Hist-FiLM under three DLO-surface conditions. Columns show steps 0, 3, and 10. Colored curves denote target shapes, and values report mean-node shape error.

From Figure 4: Zero-shot real-robot evaluation. (a) UR5 platform with a Robotiq Hand-E gripper and wrist-mounted came... — From Figure 4: Zero-shot real-robot evaluation. (a) UR5 platform with a Robotiq Hand-E gripper and wrist-mounted camera. (b) Three DLO compliance levels and the two test surfaces.

Abstract

A representation that accurately predicts a deformable object's response is not necessarily the one most useful for choosing manipulation actions. We study this distinction between predictive utility and decision utility in planar single-grasp shape control of Deformable Linear Objects (DLOs). As a common decision backbone, we introduce Geometry-Pointer-Flow (GPF), a geometry-grounded policy that separates grasp-location selection from grasp-conditioned motion generation. Its geometry-only variant reduces one-step executed shape error by 32.8% relative to a DLO-specific diffusion policy, while using 18.7 times fewer parameters and achieving approximately 193 times faster inference. On this backbone, we compare physical context represented by ground-truth parameters, estimated parameters, and learned interaction-history representations, for both forward prediction and executed control. For prediction, ground-truth physical parameters reduce next-shape error by 15.8% over geometry alone, and estimated parameters are slightly more predictive than interaction history. This ordering does not carry over to control: estimated parameters provide no detectable one-step control gain, whereas history-conditioned GPF reduces executed error by 9.9% relative to geometry-only GPF across 470 paired simulation trials. In closed loop, interaction history improves strict-threshold attainment in simulation and yields the strongest zero-shot performance on hardware. All policies are trained exclusively in simulation and transferred to the physical robot without real-world training or fine-tuning. These results show that physical context should be evaluated by the decisions it supports rather than by predictive accuracy alone. Code will be released upon acceptance.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong