Fly0: Decoupling Semantic Grounding from Geometric Planning for Zero-Shot Aerial Navigation

2026-02-02

arXiv.org

AI Summary

Existing vision-language navigation methods struggle to balance semantic understanding with control precision, particularly when employing multimodal large language models (MLLMs) as controllers, often suffering from high latency, trajectory oscillation, and insufficient geometric awareness. This work proposes Fly0, a framework that decouples semantic reasoning from geometric planning through a three-stage navigation pipeline: MLLM-based mapping of instructions to 2D coordinates, depth-informed 3D goal localization, and metric-space obstacle-aware trajectory planning. Fly0 establishes the first persistent mapping from natural language instructions to metric anchors, enabling robust navigation even under visual degradation while substantially reducing computational overhead and instability. Experiments demonstrate that Fly0 improves navigation success rates by over 20% and reduces path errors by approximately 50% in both simulated and real-world unstructured environments.

Abstract

Current Visual-Language Navigation (VLN) methodologies face a trade-off between semantic understanding and control precision. While Multimodal Large Language Models (MLLMs) offer superior reasoning, deploying them as low-level controllers leads to high latency, trajectory oscillations, and poor generalization due to weak geometric grounding. To address these limitations, we propose Fly0, a framework that decouples semantic reasoning from geometric planning. The proposed method operates through a three-stage pipeline: (1) an MLLM-driven module for grounding natural language instructions into 2D pixel coordinates; (2) a geometric projection module that utilizes depth data to localize targets in 3D space; and (3) a geometric planner that generates collision-free trajectories. This mechanism enables robust navigation even when visual contact is lost. By eliminating the need for continuous inference, Fly0 reduces computational overhead and improves system stability. Extensive experiments in simulation and real-world environments demonstrate that Fly0 outperforms state-of-the-art baselines, improving the Success Rate by over 20\% and reducing Navigation Error (NE) by approximately 50\% in unstructured environments. Our code is available at https://github.com/xuzhenxing1/Fly0.

Problem

• Visual-Language Navigation
• Multimodal Large Language Models
• geometric grounding
• trajectory oscillations
• control precision

Innovation

• zero-shot navigation
• multimodal large language models
• geometric planning
• aerial vision-language navigation
• metric anchoring