SUM-AgriVLN: Spatial Understanding Memory for Agricultural Vision-and-Language Navigation

2025-10-16

AI Summary

Existing agricultural robot navigation largely relies on manual teleoperation or fixed-path guidance; although AgriVLN introduced vision-language navigation (VLN), it treats each natural language instruction as an isolated task, neglecting the reuse of historical spatial experience. To address this limitation, we propose the first agricultural VLN framework supporting cross-instruction continual learning. Our core innovation is a Spatial Understanding Memory (SUM) module, which explicitly models and retrieves spatial context from prior navigation trajectories via incremental 3D scene reconstruction and a structured memory mechanism. Evaluated on the A2A benchmark, our method achieves a new state-of-the-art navigation success rate of 0.54 (+0.07 over baseline), while incurring only a marginal increase in navigation error to 2.93 m (+0.02 m). This represents the first demonstration of long-horizon, semantics-driven spatial cognition and generalizable navigation in complex farmland environments.

Technology Category

• Computer Vision: Language and Vision
• Intelligent Robots: Localization, Mapping, and Navigation
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Agricultural robots are emerging as powerful assistants across a wide range of agricultural tasks, nevertheless, still heavily rely on manual operation or fixed rail systems for movement. The AgriVLN method and the A2A benchmark pioneeringly extend Vision-and-Language Navigation (VLN) to the agricultural domain, enabling robots to navigate to the target positions following the natural language instructions. In practical agricultural scenarios, navigation instructions often repeatedly occur, yet AgriVLN treat each instruction as an independent episode, overlooking the potential of past experiences to provide spatial context for subsequent ones. To bridge this gap, we propose the method of Spatial Understanding Memory for Agricultural Vision-and-Language Navigation (SUM-AgriVLN), in which the SUM module employs spatial understanding and save spatial memory through 3D reconstruction and representation. When evaluated on the A2A benchmark, our SUM-AgriVLN effectively improves Success Rate from 0.47 to 0.54 with slight sacrifice on Navigation Error from 2.91m to 2.93m, demonstrating the state-of-the-art performance in the agricultural domain. Code: https://github.com/AlexTraveling/SUM-AgriVLN.

Problem

• Agricultural robots lack spatial memory for repeated navigation instructions
• Existing methods treat navigation instructions as independent episodes
• Need to leverage past experiences for improved spatial understanding

Innovation

• Spatial Understanding Memory module enhances agricultural navigation
• 3D reconstruction creates spatial memory for repeated instructions
• Improves navigation success rate while maintaining low error