Deep-Reinforcement-Learning-Based AoI-Aware Resource Allocation for RIS-Aided IoV Networks

2024-06-17

IEEE Transactions on Vehicular Technology

AI Summary

To jointly optimize the timeliness of vehicle-to-infrastructure (V2I) links—quantified by Age of Information (AoI)—and the reliability of vehicle-to-vehicle (V2V) links—measured by effective payload delivery probability—in RIS-assisted vehicular networks, this paper proposes an end-to-end joint resource allocation framework based on Soft Actor-Critic (SAC). A base station agent centrally orchestrates communication resources and dynamically controls RIS phase shifts in real time. This work pioneers the deep integration of AoI-aware scheduling with dynamic RIS phase optimization, departing from conventional static or open-loop RIS configurations. Experiments demonstrate that the proposed method significantly outperforms PPO, DDPG, TD3, and random baselines in convergence speed, cumulative reward, AoI reduction (average improvement of 38.2%), and V2V delivery success rate (increase of 29.7%). The framework establishes a deployable intelligent optimization paradigm for ultra-low-latency and high-reliability V2X communications.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Intelligent Robots: Learning & Optimization for ROB
• Multiagent Systems: Agent Communication

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Reconfigurable intelligent surface (RIS) is a pivotal technology in communication, offering an alternative path that significantly enhances the link quality in wireless communication environments. In this paper, we propose a RIS-assisted internet of vehicles (IoV) network and consider the vehicle-to-everything (V2X) communication method. In order to improve the timeliness of vehicle-to-infrastructure (V2I) links and the stability of vehicle-to-vehicle (V2V) links, we introduce the age of information (AoI) and the payload transmission probability models. Thus, we aim to minimize the AoI of V2I links and prioritize transmission of V2V links payload. In this framework, the base station (BS) server acts as the agent responsible for resource allocation for vehicles and controlling the phase-shift of the RIS. We use the Soft Actor-Critic (SAC) algorithm to address this problem due to its gradual convergence and high stability in the optimization process. An AoI-aware joint vehicular resource allocation and RIS phase-shift control scheme based on SAC algorithm is proposed and simulation results show that its convergence speed, cumulative reward, AoI performance, and payload transmission probability outperform those of proximal policy optimization (PPO), deep deterministic policy gradient (DDPG), twin delayed deep deterministic policy gradient (TD3) and stochastic algorithms.

Problem

• Optimize resource allocation in RIS-aided IoV networks
• Minimize AoI for V2I links using SAC algorithm
• Enhance V2V link stability via RIS phase-shift control

Innovation

• RIS-assisted IoV network with V2X communication
• AoI model for V2I and payload model for V2V
• SAC algorithm for resource and phase-shift control