T-araVLN: Translator for Agricultural Robotic Agents on Vision-and-Language Navigation

2025-09-08

AI Summary

Existing agricultural vision-language navigation (VLN) methods exhibit limited comprehension of complex natural-language instructions, resulting in low task success rates and substantial localization errors in real-world farmland environments. To address this, we propose AgriVLN+, a novel VLN framework for agriculture that introduces an instruction translation module to transform ambiguous and redundant raw instructions into semantically precise, structurally explicit navigation directives. The framework further integrates multi-modal perception (RGB-D, LiDAR, and GPS) with optimized path planning to enhance robustness and accuracy. Evaluated on the agriculture-specific A2A benchmark, AgriVLN+ achieves a task success rate of 0.63—up from 0.47—and reduces average navigation error from 2.91 m to 2.28 m, establishing new state-of-the-art performance in agricultural VLN. This work is the first to systematically resolve the challenges of semantic parsing and robust execution of complex agronomic instructions.

Technology Category

• Computer Vision: Language and Vision
• Intelligent Robots: Localization, Mapping, and Navigation
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Agricultural robotic agents have been becoming powerful helpers in a wide range of agricultural tasks, nevertheless, still heavily rely on manual operation or untransportable railway for movement. The AgriVLN method and the A2A benchmark pioneeringly extend Vision-and-Language Navigation (VLN) to the agricultural domain, enabling agents navigate to the target position following the natural language instructions. AgriVLN effectively understands the simple instructions, however, often misunderstands the complicated instructions. To bridge this gap, we propose the method of Translator for Agricultural Robotic Agents on Vision-and-Language Navigation (T-araVLN), in which the Instruction Translator module translates the original instruction to be both refined and precise. Being evaluated on the A2A benchmark, our T-araVLN effectively improves SR from 0.47 to 0.63 and reduces NE from 2.91m to 2.28m, demonstrating the state-of-the-art performance in the agricultural domain. Code: https://github.com/AlexTraveling/T-araVLN.

Problem

• Agricultural robots struggle with complex navigation instructions
• Existing methods misunderstand complicated language commands
• Need for precise instruction translation in agricultural VLN

Innovation

• Instruction Translator module refines natural language
• Improves navigation precision for agricultural robotic agents
• Achieves state-of-the-art performance on A2A benchmark