A Lyapunov-Guided Diffusion-Based Reinforcement Learning Approach for UAV-Assisted Vehicular Networks with Delayed CSI Feedback

2025-07-28

AI Summary

This work addresses the problem of optimizing communication rate in UAV-assisted vehicular networks under channel state information (CSI) feedback delay and long-term energy constraints. To this end, we propose a joint resource management framework integrating Lyapunov optimization with diffusion models, enabling slot-level coordinated decisions on channel allocation, transmit power, and UAV flight altitude. Our key contribution is a novel deep deterministic policy gradient (D3PG) algorithm incorporating a generative diffusion prior, which decomposes the global energy constraint into instantaneous, tractable local control objectives while ensuring long-term UAV energy sustainability. Extensive simulations based on real-world vehicle trajectories demonstrate that the proposed method significantly outperforms existing baselines in both aggregate communication rate and energy efficiency. The framework establishes a scalable, intelligent resource orchestration paradigm for highly dynamic air-ground integrated vehicular communications in the emerging low-altitude economy.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Low altitude uncrewed aerial vehicles (UAVs) are expected to facilitate the development of aerial-ground integrated intelligent transportation systems and unlocking the potential of the emerging low-altitude economy. However, several critical challenges persist, including the dynamic optimization of network resources and UAV trajectories, limited UAV endurance, and imperfect channel state information (CSI). In this paper, we offer new insights into low-altitude economy networking by exploring intelligent UAV-assisted vehicle-to-everything communication strategies aligned with UAV energy efficiency. Particularly, we formulate an optimization problem of joint channel allocation, power control, and flight altitude adjustment in UAV-assisted vehicular networks. Taking CSI feedback delay into account, our objective is to maximize the vehicle-to-UAV communication sum rate while satisfying the UAV's long-term energy constraint. To this end, we first leverage Lyapunov optimization to decompose the original long-term problem into a series of per-slot deterministic subproblems. We then propose a diffusion-based deep deterministic policy gradient (D3PG) algorithm, which innovatively integrates diffusion models to determine optimal channel allocation, power control, and flight altitude adjustment decisions. Through extensive simulations using real-world vehicle mobility traces, we demonstrate the superior performance of the proposed D3PG algorithm compared to existing benchmark solutions.

Problem

• Optimize UAV-assisted vehicular networks with delayed CSI feedback
• Maximize communication sum rate under UAV energy constraints
• Develop diffusion-based reinforcement learning for resource and trajectory optimization

Innovation

• Lyapunov optimization for problem decomposition
• Diffusion-based D3PG algorithm for decision-making
• Joint channel, power, and altitude optimization