VerifLM: Physically Verified Image-Edit-Guided Goal Pose Generation for Robotic Manipulation

Xuanyou Chen, Jinyu Chen, Glen Chou

November 2026

TL;DR VerifLM edits a scene image to show the goal, recovers a 6-DoF object pose from the edit, then searches around it and checks candidates in MuJoCo digital twins. On nine real KUKA tasks it reaches 74.1% task-planning success, against 47.2% for the best baseline (Goal-VLA).

What they do

Many placement goals, such as a plate in a specific slot of a stand, are easy to picture but hard to write down as poses or keypoint constraints. VerifLM asks an image-editing model to draw the goal, recovers the active object’s pose relative to the passive object from that image, and treats it only as a starting guess. A VLM decides where to search around it, and a physics simulator decides which candidates are actually valid.

How it works

Goal from the edit. VGGT estimates depth for the source and edited images, rescaled to the real RGB-D depth; GroundingDINO, SAM2 and FoundationPose give the object poses, and only the relative pose is kept.
VLM-guided search. The VLM marks which translation and rotation axes need correcting, and by how much, and writes a task-specific validation program: required and forbidden contacts, allowed drift and position relations.
Simulation check. Candidates settle under gravity in MuJoCo. Without known meshes, Hunyuan3D reconstructs the objects and each candidate is tested in 27 parallel worlds (3 scales per object × 3 VLM-guessed mass/friction sets). The VLM picks among the most robust; AnyGrasp and RRTConnect execute.

Results

Real KUKA iiwa, nine tasks × 24 trials: with known meshes, 160/216 (74.1%) task-planning and 155/216 (71.8%) execution success, versus 102/216 for Goal-VLA, 95 for OmniManip and 71 for ReKep. Best on 8 of 9 tasks; Pour Tea is the exception.
Mesh-free: 146/216 planning, 130/216 execution success.
Using the image-derived pose directly succeeds on 84/216; parallel worlds raise mesh-free execution from 92/216 to 130/216.

Limitations

Performance depends on reconstruction quality: missing or distorted contact features break the digital twin. Variable VLM edits and reasoning can produce inconsistent search ranges or validation programs.

Videos

Figures

From Figure 1: ReKep and OmniManip goals compared with VerifLM. All work when orientation barely matters (mug on plat... — From Figure 1: ReKep and OmniManip goals compared with VerifLM. All work when orientation barely matters (mug on plate), but the baselines fail when the goal needs a specific orientation or contact, as in plate on stand.

From Figure 3: Overview of VerifLM, which turns an edited goal image into a physically verified object goal pose and ... — From Figure 3: Overview of VerifLM, which turns an edited goal image into a physically verified object goal pose and executes a trajectory to reach it.

From Figure 5: Two error sources in the image-derived pose: the image editor lengthens the rack branch into an infeas... — From Figure 5: Two error sources in the image-derived pose: the image editor lengthens the rack branch into an infeasible contact, and VGGT plus FoundationPose overestimate the mug-rack distance.

Abstract

Many manipulation goals are easier to visualize than to specify numerically through explicit geometric constraints. We present VerifLM, a framework that converts image-edited goal configurations into physically verified 6-DoF object poses for planning. Using an edited image, VerifLM recovers an initial relative pose between an active object and a passive object, then prompts a vision-language model to generate axis-wise sampling constraints and a task-specific validation program. Candidate poses are evaluated in a physics simulator, and the VLM selects the most task-consistent retained candidate. To operate without predefined object meshes or physical parameters, VerifLM reconstructs metric object geometry from RGB-D observations and constructs multiple hypothetical simulations with varying physics and geometry. Parallel-world validation evaluates candidates across geometry--physics variations and ranks them by the number of worlds in which they satisfy the validation program. We evaluate VerifLM on nine manipulation tasks. VerifLM achieves the highest task success on eight of nine tasks, with the largest performance gain on goals requiring precise orientation and contact relationships.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong