Matt Strong

Matt Strong

CS PhD at Stanford University

← Paper Digest

CSG-Nav: Confidence-Aware Semantic-Geometric Fusion for Robust Exploration and Accurate Zero-Shot Object-Goal Navigation

Shoubin Chen, Rui Xu, Zehao Wu, Kai Zhang, Tao Huang, Zhou Liu, Qingquan Li

November 2026

TL;DR CSG-Nav finds named objects in unseen buildings without task-specific training. It fuses YOLO-World detections, VLM scene descriptions and LLM scoring into a distance-weighted map, and stops only once depth places the target within 1 m. With 4B-parameter models it reaches 58.9% and 71.6% success on HM3Dv1 and HM3Dv2, and 25 of 50 real-robot trials.

What they do

In zero-shot object-goal navigation, a robot must find a named object such as “a sofa” in an unseen building, with no task-specific training. Methods built on foundation models often feed VLM scores straight into planning. Hallucinated or distant cues then steer exploration, and the robot stops as soon as it recognizes the target, even when the target is still metres away. CSG-Nav fuses detector, VLM and depth evidence at three stages, using only 4B-parameter Qwen models.

How it works

  • Semantic-source fusion. At each step the robot views six directions. For each direction, YOLO-World detections with their confidences and a Qwen3-VL-4B scene description go to Qwen3-4B, which scores the direction from 0 to 10 for the target.
  • Spatial-confidence fusion. Scores are projected onto accessible map regions, weighted by a Gaussian that decays with distance from the robot, and merged with earlier views into a preference map that picks the next direction. The VLM then chooses among candidate moves drawn as numbered arrows on the image.
  • Semantic-geometric stopping. When the target is detected (confidence ≥ 0.25), its 3D position comes from the box center and depth, averaged over three frames. The robot stops only within 1.0 m.

Results

  • Habitat ObjectNav success: 58.9% on HM3Dv1 and 71.6% on HM3Dv2, versus 58.1% and 69.8% for WMNav with Gemini 1.5 Pro; on MP3D, 44.3% versus 45.4%. SPL is lower than some baselines (24.5 vs 31.2 on HM3Dv1), which the authors attribute to extra approach moves before stopping.
  • Ablations on 100 HM3Dv2 episodes: Gaussian weighting lifts success from 69 to 71, adding VLM descriptions to the detections from 71 to 75, and depth-based stopping from 75 to 77.
  • Real robot (wheeled base, RGB-D camera, all models onboard a Jetson AGX Orin): 15/25 trials succeed indoors and 10/25 outdoors across ten targets, ranging from 5/5 for a water cooler to 1/5 for a toilet.

Limitations

The distance prior and the map initialization are hand-set, and the initial preference value strongly affects performance. Perception is unreliable for high, occluded or out-of-view targets. The real-world tests include no baselines or ablations on the same robot, and the ablations are single runs.

Figures

From Figure 1: Semantic-only navigation trusts unreliable long-range evidence and stops falsely when it merely recogn...
From Figure 1: Semantic-only navigation trusts unreliable long-range evidence and stops falsely when it merely recognizes the target. CSG-Nav counters these with distance-decayed confidence fusion and semantic-geometric arrival verification.
From Figure 2: Overview of CSG-Nav. Semantic-source fusion, spatial-confidence fusion and semantic-geometric decision...
From Figure 2: Overview of CSG-Nav. Semantic-source fusion, spatial-confidence fusion and semantic-geometric decision fusion feed exploration planning and the stop check.
From Figure 8: Navigation trajectories on HM3Dv2, CSG-Nav in green and WMNav in red.
From Figure 8: Navigation trajectories on HM3Dv2, CSG-Nav in green and WMNav in red.
From Figure 10: Real-world runs from start to goal: an elevator and a water cooler indoors, a trash can outdoors. Num...
From Figure 10: Real-world runs from start to goal: an elevator and a water cooler indoors, a trash can outdoors. Numbered candidate actions are drawn on each view.
Abstract

Zero-shot object-goal navigation requires autonomous agents to explore unseen environments and locate target objects without task-specific training, posing challenges in perceptual reliability under observation uncertainty and physically grounded goal termination. Although foundation-model-based approaches improve generalization, they remain vulnerable to unreliable long-range perception, hallucination-induced directional bias, and semantically plausible but geometrically incorrect stopping decisions. To address these limitations, we propose CSG-Nav, a zero-shot navigation framework centered on confidence-aware multimodal information fusion. CSG-Nav integrates three complementary fusion mechanisms. First, semantic-source fusion integrates open-vocabulary detection results, detection confidence, scene observations, and relational reasoning to produce more reliable direction-level semantic scores and suppress hallucination-induced directional bias. Second, spatial-confidence fusion weights multi-view semantic evidence using a Gaussian decay function, suppressing unreliable distant observations while accumulating spatially consistent exploration preferences. Third, semantic-geometric decision fusion combines semantic target detection with depth-based 3D localization to support physically grounded stopping decisions. Experiments on HM3Dv1, HM3Dv2, and MP3D demonstrate competitive zero-shot performance, achieving success rates of 58.9%, 71.6%, and 44.3%. Fusion-level ablations and real-world deployments in indoor and outdoor environments further verify the effectiveness and practicality of CSG-Nav for robust navigation under uncertain perception. Code is available at https://xr-ui.github.io/CSGNav.github.io/

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.