Offline Vision-Language Navigation with Geometric Goal Localization for Outdoor Environments

2026-07-24

AI Summary

This work addresses the limitations of existing vision-language navigation systems, which rely on cloud-based large models and are thus impractical in outdoor environments lacking network connectivity or requiring high-precision localization. We propose Edge-BehAV, the first fully edge-deployable vision-language navigation system, which employs a lightweight language model to parse natural language instructions and integrates open-vocabulary object detection, prompt-driven segmentation, and LiDAR-based geometric modeling into a unified semantic-geometric target localization framework. We present the first systematic evaluation of 17 edge-compatible small language models for instruction decomposition, identifying a top-performing model that matches the accuracy of the strongest cloud API while achieving 9× faster inference. Our approach reduces target localization error from 2.05 meters to 0.20 meters and succeeds in 31 out of 32 real-world outdoor closed-loop navigation trials.

Abstract

Foundation-model-based vision-language navigation (VLN) has advanced autonomous robot navigation by enabling robots to interpret natural-language instructions, identify semantic goals, and follow user-specified behavioral rules. However, existing VLN systems rely heavily on cloud-hosted foundation models for language understanding and semantic grounding, limiting their applicability where network connectivity is unavailable and reliable metric goal localization is required. Although recent small language models (SLMs) enable fully onboard inference, their suitability for navigation instruction decomposition has not been systematically evaluated. This paper makes three contributions toward fully onboard VLN for outdoor environments. First, we present the first systematic benchmark of 17 edge-deployable SLMs against 4 online APIs for robotic navigation instruction decomposition, evaluating accuracy and latency on human-annotated instructions across three computing platforms and providing practical guidance for selecting onboard language models. Second, we propose a lightweight hybrid semantic-geometric goal localization framework that combines open-vocabulary object detection, prompted segmentation, and LiDAR geometry to estimate metric goals, while maintaining visual bearing guidance when reliable geometric observations are unavailable. Third, we integrate these advances into Edge-BehAV, a fully onboard extension of the BehAV architecture that enables cloud-independent behavior-guided navigation. Experimental results show that the best offline SLM matches the instruction decomposition performance of the strongest cloud API while running approximately 9x faster and without network connectivity. The proposed goal localization framework reduces mean goal-distance error from 2.05 m to 0.20 m at lower computational cost, and the complete system succeeds in 31 of 32 closed-loop outdoor trials.

Problem

• vision-language navigation
• offline navigation
• goal localization
• small language models
• outdoor environments

Innovation

• small language models
• onboard navigation
• semantic-geometric localization
• vision-language navigation
• edge computing