Can LLM be a Good Path Planner based on Prompt Engineering? Mitigating the Hallucination for Path Planning

2024-08-23

arXiv.org

AI Summary

Large language models (LLMs) face dual challenges in maze path planning: spatial hallucination and contextual inconsistency during long-range reasoning. To address these, we propose the Spatial-to-Relational Conversion and Curriculum Q-Learning (S2RCQL) framework. Our method introduces a novel “spatial → relational” prompting paradigm that maps geometric coordinates to topological relations—thereby mitigating spatial hallucination—and integrates Q-value-augmented prompts within a curriculum-based Q-learning scheme. Furthermore, we design an LLM-driven inverse curriculum learning mechanism to alleviate context drift. Experiments on ERNIE-Bot 4.0 demonstrate that S2RCQL improves path planning success rate and optimality by 23%–40% over state-of-the-art prompting approaches. This work establishes a new paradigm for modeling LLMs’ spatial reasoning capabilities through structured relational abstraction and adaptive reinforcement-informed prompting.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Intelligent Robots: Motion and Path Planning
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Spatial reasoning in Large Language Models (LLMs) is the foundation for embodied intelligence. However, even in simple maze environments, LLMs still encounter challenges in long-term path-planning, primarily influenced by their spatial hallucination and context inconsistency hallucination by long-term reasoning. To address this challenge, this study proposes an innovative model, Spatial-to-Relational Transformation and Curriculum Q-Learning (S2RCQL). To address the spatial hallucination of LLMs, we propose the Spatial-to-Relational approach, which transforms spatial prompts into entity relations and paths representing entity relation chains. This approach fully taps the potential of LLMs in terms of sequential thinking. As a result, we design a path-planning algorithm based on Q-learning to mitigate the context inconsistency hallucination, which enhances the reasoning ability of LLMs. Using the Q-value of state-action as auxiliary information for prompts, we correct the hallucinations of LLMs, thereby guiding LLMs to learn the optimal path. Finally, we propose a reverse curriculum learning technique based on LLMs to further mitigate the context inconsistency hallucination. LLMs can rapidly accumulate successful experiences by reducing task difficulty and leveraging them to tackle more complex tasks. We performed comprehensive experiments based on Baidu's self-developed LLM: ERNIE-Bot 4.0. The results showed that our S2RCQL achieved a 23%--40% improvement in both success and optimality rates compared with advanced prompt engineering.

Problem

• Addressing LLMs' spatial hallucination in path planning
• Mitigating context inconsistency in long-term reasoning
• Improving path-planning success and optimality rates

Innovation

• Spatial-to-Relational Transformation for spatial hallucination
• Q-learning algorithm for context inconsistency hallucination
• Reverse curriculum learning to enhance task complexity handling