SkillNav: Score-Level Skill Intervention for Zero-Shot Object Goal Navigation

2026-07-17

AI Summary

This work addresses the limited cross-step behavioral awareness of existing vision-language models (VLMs) in zero-shot object navigation, which often leads to dead ends or ineffective loops. The authors propose SkillNav, a novel framework that introduces a score-level, three-tiered skill intervention mechanism—comprising soft scaling, lower-bound boosting, and hard overriding—to inject composable behavioral skills into the VLM’s intrinsic curiosity value map at zero token cost. This enables plug-and-play deployment and continual evolution without any retraining. By integrating spatial memory (via value map writing) and semantic memory (through short prompts) in a dual-representation design, SkillNav achieves new state-of-the-art SPL scores of 25.5, 39.3, and 43.2 on MP3D and HM3D (v0.1/v0.2), with absolute gains up to 6.0, and sets the highest success rates on HM3D (69.7% for v0.1 and 75.9% for v0.2).

Abstract

Vision-Language Model (VLM) agents have advanced zero-shot object-goal navigation, yet single-frame reasoning leaves them without the cross-step behavioral awareness an embodied navigator requires, producing recurring failures such as dead-end stalls, in-room loops, and circuitous approaches to detected targets. Prompt-based remedies inflate token budgets across multi-submodule episodes and still struggle to encode inherently spatial signals such as angles, map cells, and viewpoint coordinates. In this paper, we propose SkillNav, an extensible behavioral skill framework for VLM-based navigation that treats the curiosity value map already maintained by modern VLM navigators as a writable substrate on which composable skills inscribe behavioral memory at zero token cost. Skills are stratified into three tiers by their level of behavioral authority, namely soft scaling for proportional reweighting, lower-bound boost for region-level guarantees, and hard override for threshold-triggered forced actions, and cooperate across tiers under a fixed composition order that establishes a predictable, declared priority among skills. This design turns capability improvement into skill registration: new behaviors plug in without retraining the VLM or disturbing existing skills, opening a path for continual refinement. A minimal prompt channel complements the score-level skills with category-level semantic hints, yielding a dual-representation design in which spatial memory lives on the map and semantic memory in short prompts. Training-free, SkillNav establishes new state-of-the-art SPL across MP3D (25.5), HM3D v0.1 (39.3), and HM3D v0.2 (43.2), improving SPL by up to 6.0 absolute over the strongest prior method, and achieves the highest Success Rate on HM3D v0.1 (69.7) and v0.2 (75.9).

Problem

• zero-shot object-goal navigation
• behavioral awareness
• spatial signals
• token budget
• embodied navigation

Innovation

• Score-level intervention
• Behavioral skill framework
• Zero-shot navigation
• Curiosity value map
• Dual-representation design