MemVLN: Episodic and Procedural Memory for Vision-and-Language Navigation

2026-07-26

AI Summary

This work addresses the challenge in vision-and-language navigation within continuous environments, where existing approaches struggle to simultaneously maintain long-term visual memory and achieve low-latency action execution. To bridge this gap, the authors propose MemVLN, a novel framework that integrates multi-resolution episodic memory with procedural memory grounded in a compact vocabulary of atomic mid-level actions. The episodic memory module compresses and retains critical historical perceptual information to ensure long-term consistency, while the procedural memory enables efficient real-time decision-making through non-autoregressive generation. By synergistically combining a visual encoder with a large language model, MemVLN achieves significant performance gains—improving success rates by 5.8% on R2R and 9.7% on RxR—while attaining an inference speed of 14 FPS, representing a sevenfold acceleration over baseline methods.

Abstract

Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires agents to maintain long-horizon visual history for trajectory consistency while executing actions with low latency. Existing video-based VLN approaches typically struggle to satisfy both demands simultaneously. To address these challenges, we propose MemVLN, a novel VLN framework that achieves state-of-the-art performance with real-time inference efficiency (14 FPS). MemVLN utilizes a visual encoder to process continuous observations and a Large Language Model (LLM) to interpret instructions and generate actions. Central to our approach is an Episodic Memory management that applies pyramidal resolutions. This mechanism concentrates computation on immediate percepts while retaining compressed long-term history. Complementing to this design, we introduce Procedural Memory for fast action with a compact vocabulary of atomic mid-level actions to bypass auto-regressive decoding latency. Experiments on VLN-CE show that MemVLN-4B surpasses the baseline Qwen3-VL-4B architecture by 5.8\% SR in R2R and 9.7\% SR in RxR, while achieving a 7$\times$ speedup in inference latency.

Problem

• Vision-and-Language Navigation
• Continuous Environments
• Long-horizon Memory
• Low-latency Execution
• Trajectory Consistency

Innovation

• Episodic Memory
• Procedural Memory
• Vision-and-Language Navigation
• Real-time Inference
• Pyramidal Resolution