Path Planning using Instruction-Guided Probabilistic Roadmaps

2025-02-23

AI Summary

To address the inflexibility and manual effort required in encoding task constraints for mobile robot navigation, this paper proposes a natural language-driven end-to-end path planning framework. The method integrates a large language model (LLM) to conditionally generate semantic-aware cost maps—its first application in this context—and tightly couples this with an enhanced probabilistic roadmap (PRM) and A* search. An instruction-guided conditional diffusion/regression model enables direct mapping from natural language instructions to semantic cost maps and, subsequently, to safe, executable paths. Experiments on both synthetic and real-world indoor scenes demonstrate that the framework significantly improves instruction compliance, while achieving superior path safety and task adaptability compared to conventional PRM and state-of-the-art learning-based baselines.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Intelligent Robots: Motion and Path Planning
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

This work presents a novel data-driven path planning algorithm named Instruction-Guided Probabilistic Roadmap (IG-PRM). Despite the recent development and widespread use of mobile robot navigation, the safe and effective travels of mobile robots still require significant engineering effort to take into account the constraints of robots and their tasks. With IG-PRM, we aim to address this problem by allowing robot operators to specify such constraints through natural language instructions, such as ``aim for wider paths'' or ``mind small gaps''. The key idea is to convert such instructions into embedding vectors using large-language models (LLMs) and use the vectors as a condition to predict instruction-guided cost maps from occupancy maps. By constructing a roadmap based on the predicted costs, we can find instruction-guided paths via the standard shortest path search. Experimental results demonstrate the effectiveness of our approach on both synthetic and real-world indoor navigation environments.

Problem

• Develop instruction-guided path planning
• Convert natural language to cost maps
• Enhance mobile robot navigation safety

Innovation

• Instruction-guided probabilistic roadmap
• Natural language to embedding vectors
• LLM-based cost map prediction