Towards Multi-agent Policy-based Directed Hypergraph Learning for Traffic Signal Control

2024-09-08

arXiv.org

AI Summary

Traditional graph neural networks struggle to capture complex high-order spatiotemporal dependencies and non-pairwise spatial relationships in traffic networks, limiting the performance of multi-intersection cooperative traffic signal control. To address this, we propose DHG-PPO: a multi-agent proximal policy optimization framework built upon dynamic directed hypergraphs (DHGs). DHG-PPO explicitly models high-order topological dependencies among intersections via hyperedges, enabling agents to coordinate decisions under dynamically evolving road network structures. It abandons the static graph assumption, instead constructing traffic-state-driven hypergraphs in real time and performing adaptive message passing. Evaluated on multiple city-scale traffic benchmarks, DHG-PPO achieves a 18.7% reduction in average travel time and a 15.2% increase in network throughput, outperforming existing state-of-the-art methods. These results demonstrate the critical importance of high-order relational modeling for intelligent traffic signal control.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Multiagent Systems: Multiagent Planning
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Deep reinforcement learning (DRL) methods that incorporate graph neural networks (GNNs) have been extensively studied for intelligent traffic signal control, which aims to coordinate traffic signals effectively across multiple intersections. Despite this progress, the standard graph learning used in these methods still struggles to capture higher-order correlations in real-world traffic flow. In this paper, we propose a multi-agent proximal policy optimization framework DHG-PPO, which incorporates PPO and directed hypergraph module to extract the spatio-temporal attributes of the road networks. DHG-PPO enables multiple agents to ingeniously interact through the dynamical construction of hypergraph. The effectiveness of DHG-PPO is validated in terms of average travel time and throughput against state-of-the-art baselines through extensive experiments.

Problem

• Capturing intricate spatio-temporal traffic correlations
• Modeling complex directed relationships in road networks
• Enabling adaptive traffic signal control decisions

Innovation

• Multi-agent policy-based directed hypergraph learning
• Dynamic directed hypergraph construction mechanism
• Adaptive decision-making in traffic signal control