Hybrid LLM-DDQN based Joint Optimization of V2I Communication and Autonomous Driving

2024-10-11

arXiv.org

AI Summary

This study addresses the challenges of tight coupling, high latency, and frequent handovers between V2I communication and autonomous driving decision-making in cooperative vehicle-infrastructure systems. We propose a joint optimization framework integrating Large Language Models (LLMs) and Deep Double Q-Networks (DDQN) in an iterative manner. Our method encodes LLM-generated semantic driving policies as structured V2I state inputs to enable cross-modal policy coordination; introduces a Euclidean distance–driven historical experience retrieval mechanism to enhance LLM’s online adaptability; and constructs a unified state space with an iterative policy optimization pipeline. Experimental results demonstrate that, compared to a pure DDQN baseline, our approach achieves a 37% faster convergence rate and a 22% higher average reward, while significantly improving communication stability, data throughput, and driving safety.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Search and Optimization: Learning to Search
• Multiagent Systems: Agent Communication

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large language models (LLMs) have received considerable interest recently due to their outstanding reasoning and comprehension capabilities. This work explores applying LLMs to vehicular networks, aiming to jointly optimize vehicle-to-infrastructure (V2I) communications and autonomous driving (AD) policies. We deploy LLMs for AD decision-making to maximize traffic flow and avoid collisions for road safety, and a double deep Q-learning algorithm (DDQN) is used for V2I optimization to maximize the received data rate and reduce frequent handovers. In particular, for LLM-enabled AD, we employ the Euclidean distance to identify previously explored AD experiences, and then LLMs can learn from past good and bad decisions for further improvement. Then, LLM-based AD decisions will become part of states in V2I problems, and DDQN will optimize the V2I decisions accordingly. After that, the AD and V2I decisions are iteratively optimized until convergence. Such an iterative optimization approach can better explore the interactions between LLMs and conventional reinforcement learning techniques, revealing the potential of using LLMs for network optimization and management. Finally, the simulations demonstrate that our proposed hybrid LLM-DDQN approach outperforms the conventional DDQN algorithm, showing faster convergence and higher average rewards.

Problem

• V2I Optimization
• Autonomous Driving
• Network Efficiency

Innovation

• LLM-DDQN
• V2I-Communication-Optimization
• Autonomous-Driving-Decision-Enhancement