VLN-AVP: Zero-Shot Vision-Language Navigation with Hybrid Long-Short-Term Memory for Autonomous Valet Parking

2026-07-20

AI Summary

This work addresses the limitations of conventional automated valet parking systems, which rely on pre-built maps and struggle in unknown environments or with open-vocabulary natural language instructions. We propose VLN-AVP, a zero-shot navigation framework that integrates bird’s-eye-view (BEV) spatial perception with visual-language model (VLM) semantic understanding. Our approach introduces a novel hybrid memory architecture combining short-term perceptual memory with long-term topological memory, enabling map-free, language-guided autonomous parking. To support this research, we present the first visual-language navigation dataset and benchmark for underground parking lots. In simulation, VLN-AVP outperforms existing vision-and-language navigation methods by over 25% in success rate and surpasses other autonomous driving approaches by more than 15%. Real-world vehicle experiments further validate its superior performance.

Abstract

Existing methods in Autonomous Valet Parking (AVP) typically rely on pre-built maps, which severely restricts their scalability to unseen environments and open-vocabulary targets. Inspired by the application of Vision-Language Models (VLMs) in Vision-Language Navigation (VLN) tasks, we propose VLN-AVP, a zero-shot navigation framework for AVP tasks. By combining the precise spatial perception of a Bird's-Eye-View (BEV) model with the general intelligence of VLMs, our framework 1) eliminates the dependency on pre-built maps, 2) interprets semantic environmental contexts in parking scenarios, and 3) enables intuitive navigation following natural language instructions. Specifically, we introduce a hybrid memory system: a short-term perception memory tracks semantic visual cues to address the limitations of VLM's single-frame reasoning in existing methods, while a long-term topological memory facilitates stable policy learning from past experiences. To bridge the gap in existing benchmarks, we also present the VLN-AVP dataset and benchmark. Featuring 10 high-fidelity parking scenes and over 1,000 navigation episodes, it has the largest number of garage scenes to date and is the first VLN benchmark for underground parking. Extensive experiments demonstrate that in simulation, our method achieves an over 25% improvement in success rate compared to VLN methods and an over 15% improvement compared to other autonomous driving methods. Furthermore, it attains a leading success rate in real-world vehicle experiments, proving its practical feasibility.

Problem

• Autonomous Valet Parking
• Vision-Language Navigation
• Zero-Shot Navigation
• Open-Vocabulary Targets
• Pre-built Maps

Innovation

• Zero-Shot Navigation
• Vision-Language Model
• Hybrid Memory
• Autonomous Valet Parking
• Bird's-Eye-View