Decide Before You Move: Diffusion-Generated Search Reference for Zero-Shot Open-Vocabulary Object Navigation in Semi-Static Environments

TL;DR PreNav plans an object search before the robot moves. An LLM scores which furniture the target is likely to be near, and a diffusion model turns that score and the wall map into one ordered search path. It beats SGM on Gibson ObjectNav when both get the same priors, matches its TSP teacher while generating paths 25.8x faster, and finds open-vocabulary targets in 13 of 15 trials on a real Fetch robot.
What they do
The robot must find an object given only by name, such as “tray with fruits”. Walls and furniture rarely move but small objects do, so PreNav keeps only a map of walls and furniture boxes and plans the whole search once, before moving, with no vision-language model calls or replanning along the way.
How it works
- Furniture prior. For each query an LLM scores how useful each furniture category is for finding the target. The scores are normalized per query and spread as Gaussians over the mapped furniture boxes.
- Diffusion generator. A 5.24M-parameter transformer denoiser takes the wall map, the start pose and this prior. It outputs a 100-point centerline with a search half-width at each point.
- Training labels. It is trained on 83 ScanNet++ rooms, where a TSP-with-neighborhoods solver provides the reference paths.
- Execution. The robot follows the reference and detects the target online. On the real robot it visits viewpoints sampled near the centerline.
Results
- Gibson, six target categories (600 episodes): with the same walls and five furniture categories, PreNav beats SGM by 2.3 points in success rate and 3.0 in SPL. Annotating the remaining furniture adds 6.0 points of success.
- Never trained on Gibson, the generator nearly matches its TSP teacher (70.3% vs 71.0% success) and takes 0.995 s per path instead of 25.7 s on average.
- Most failures come from detection: 19.2% of episodes detect the wrong object, while only 5.0% never see the target.
- On a Fetch robot in a lab of over 200 square meters, with SAM 3 as the detector, PreNav succeeds in 13 of 15 trials.
Limitations
PreNav assumes the target lies on or near furniture: a quadruped robot left on the open floor was never found. It needs walls and furniture mapped in advance, small objects (book, bottle, clock, cup) are limited by detection, and the real-world test covers 15 trials in one lab.
Figures




Abstract
Open-world navigation requires a robot to locate a user-specified target in an indoor scene under open-vocabulary queries. Realistic indoor environments are semi-static: walls and furniture largely persist while smaller objects move, relocate, or vanish between visits. Prior-map systems answer queries from object poses recorded at reconstruction time, which expire as soon as objects move, whereas online methods re-derive where to look from current observations every few steps, often through expensive calls to a vision--language model. Assuming that walls and large furniture do not move, we keep their map to plan paths, but use a target--furniture co-occurrence prior to decide where to search. We present PreNav, an ObjectNav framework on semi-static environments. Rather than invoking a vision-language model at each step, PreNav generates a single global search reference before the robot moves: an LLM converts target--furniture co-occurrence into a spatial prior, a conditional diffusion model turns that prior and the persistent structure into an ordered centerline with a band around it, and the robot approaches the target once it is detected online. Experiments on a Gibson ObjectNav benchmark show that PreNav outperforms a strong online baseline under matched structural priors, using a generator trained only on room-level ScanNet++ scenes and applied to floor-level Gibson without fine-tuning. Real-world experiments under object relocation and open-vocabulary queries further validate the approach.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.
