Problem
Research questions and friction points this paper is trying to address.
Visual Language Action models
autonomous robots
real-world deployment
fine-tuning cost
execution reliability
WayFinder: Hierarchical Visual-Language-Action for Zero-Shot Waypoint Generation and Low-Level Kinematic Control
🤖 AI Summary
This study addresses the challenges of unreliable execution and prohibitive fine-tuning costs in deploying Vision-Language-Action (VLA) models by proposing a hierarchical closed-loop framework. The approach decouples high-level semantic reasoning from low-level motion control, integrating a multimodal large language model with an asynchronous lightweight policy to achieve zero-shot navigation. A core innovation lies in eliminating the need for fine-tuning; the large model is invoked on demand only upon task failure, thereby effectively balancing computational efficiency with control performance. Experimental results demonstrate that the proposed framework significantly reduces computational overhead while improving navigation success rates by 27.45%.
📝 Abstract
Visual Language Action (VLA) models offer unprecedented generalization for autonomous robots; however, their real-world deployment is frequently bottlenecked by unreliable execution and the prohibitive computational cost of fine-tuning for specific robot embodiments and tasks. To bridge this gap, we propose WayFinder, an end-to-end, closed-loop hierarchical VLA framework that circumvents the need for fine-tuning by decoupling high-level task reasoning from low-level kinematic control. WayFinder utilizes a zero-shot, offboard Multimodal Large Language Model (MLLM) policy to process linguistic context and state maps for strategic waypoint generation. Asynchronously, a lightweight, onboard policy executes real-time kinematic control at high frequency based on continuous sensor feedback. We evaluate WayFinder in Microsoft AirSim, testing on four environments of varying complexity and three MLLM scales to balance prediction efficacy with computational efficiency. Our results demonstrate that WayFinder achieves superior navigation reliability compared to baseline low-level policies. By querying the high-level MLLM only during navigation failures, WayFinder eliminates the need for fine-tuning, minimizes expensive inferences, and significantly increases navigation success rates by up to 27.45%.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Hierarchical VLA
Zero-Shot Waypoint Generation
Decoupled Control
Asynchronous Architecture
Multimodal Large Language Model
🔎 Similar Papers
No similar papers found.
