RouteRL: Multi-agent reinforcement learning framework for urban route choice with autonomous vehicles

2025-02-27

AI Summary

This study addresses the challenge of collaborative path optimization for autonomous vehicles (AVs) in mixed traffic environments with human-driven vehicles. We propose a distributed dynamic routing framework that integrates multi-agent reinforcement learning (MARL) with microscopic traffic simulation. Our approach uniquely co-simulates human driver behavior—modeled via a Logit model—with AV policies trained using MAPPO, enabling, for the first time, collaborative path optimization in human-AI mixed traffic flows. The framework is implemented within the SUMO simulation platform, supporting real-time, decentralized decision-making for AVs on realistic urban road networks. Experimental results on a representative city-scale network demonstrate a 12.3% reduction in global average travel time and a 19.7% improvement in individual AV throughput. These findings validate the effectiveness, robustness, and scalability of our distributed collaborative routing approach in complex, heterogeneous traffic scenarios.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

RouteRL is a novel framework that integrates multi-agent reinforcement learning (MARL) with a microscopic traffic simulation, facilitating the testing and development of efficient route choice strategies for autonomous vehicles (AVs). The proposed framework simulates the daily route choices of driver agents in a city, including two types: human drivers, emulated using behavioral route choice models, and AVs, modeled as MARL agents optimizing their policies for a predefined objective. RouteRL aims to advance research in MARL, transport modeling, and human-AI interaction for transportation applications. This study presents a technical report on RouteRL, outlines its potential research contributions, and showcases its impact via illustrative examples.

Problem

• Optimizes route choices for autonomous vehicles
• Integrates multi-agent reinforcement learning with traffic simulation
• Advances human-AI interaction in transportation systems

Innovation

• Multi-agent reinforcement learning framework
• Microscopic traffic simulation integration
• Autonomous vehicles route optimization