IMPRINT: Image-Conditioned Query Enrichment for Long-Tail Object Goal Navigation

2026-07-27

AI Summary

This work addresses the performance limitations of existing zero-shot object navigation methods on fine-grained, long-tailed categories due to insufficient semantic representations. We propose a plug-and-play image-augmented query framework that, for the first time, incorporates web-retrieved images into long-tailed object navigation. By leveraging pretrained vision-language models to generate image-conditioned queries, our approach enables context-aware localization within queryable semantic maps without modifying the underlying navigation policy. To evaluate navigation capabilities on fine-grained subclasses, we introduce a new benchmark, HSSD-rare, and uncover the critical impact of detection quality on navigation performance. Experiments demonstrate significant improvements in both localization accuracy and end-to-end navigation success on OVON and HSSD-rare, validating the effectiveness of image-conditioned queries in long-tailed scenarios.

Abstract

Embodied AI increasingly relies on queryable semantic maps built from pre-trained vision-language models to enable zero-shot Object Goal Navigation (ObjectNav). However, existing approaches typically depend on text-only queries, which become less reliable as semantic specificity increases toward fine-grained object categories. We introduce IMPRINT, a zero-shot plug-and-play framework that enriches textual object queries with web-sourced images to improve grounding in queryable maps. Retrieved images are encoded using a vision-language model, matched against the semantic map to produce similarity maps, and aggregated to yield context-aware localization. Notably, this requires no training or modification of the underlying navigation policy. To explicitly evaluate long-tail behavior, we present HSSD-rare, a new ObjectNav benchmark built on Habitat Synthetic Scenes and featuring semantically specific subcategories. Across both OVON and HSSD-rare, image-conditioned queries consistently improve object grounding and yield end-to-end navigation gains. Further analysis reveals that translating localization gains to navigation performance depends critically on downstream detection quality, highlighting a key systems bottleneck in long-tail embodied navigation.

Problem

• Object Goal Navigation
• long-tail
• semantic grounding
• zero-shot
• embodied AI

Innovation

• image-conditioned query
• zero-shot navigation
• semantic map grounding
• long-tail object navigation
• vision-language model