VissimRL: A Multi-Agent Reinforcement Learning Framework for Traffic Signal Control Based on Vissim

2026-01-26

AI Summary

This work proposes a modular and extensible multi-agent reinforcement learning (MARL) training framework that integrates the high-fidelity traffic simulator VISSIM, which has been underutilized in existing RL research due to its complex COM interface and the absence of standardized integration protocols. By encapsulating VISSIM’s functionality through a high-level Python API, the framework establishes a unified interaction protocol for both single- and multi-agent settings, significantly lowering the barrier to entry while preserving simulation fidelity and computational efficiency. Experimental results demonstrate that the proposed approach effectively enhances traffic performance and enables robust cooperative control among agents. This framework serves as a general-purpose bridge between high-fidelity traffic simulation and MARL, facilitating the translation of academic advances into practical industrial applications in traffic signal control.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Multiagent Systems: Agent Communication
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Traffic congestion remains a major challenge for urban transportation, leading to significant economic and environmental impacts. Traffic Signal Control (TSC) is one of the key measures to mitigate congestion, and recent studies have increasingly applied Reinforcement Learning (RL) for its adaptive capabilities. With respect to SUMO and CityFlow, the simulator Vissim offers high-fidelity driver behavior modeling and wide industrial adoption but remains underutilized in RL research due to its complex interface and lack of standardized frameworks. To address this gap, this paper proposes VissimRL, a modular RL framework for TSC that encapsulates Vissim's COM interface through a high-level Python API, offering standardized environments for both single- and multi-agent training. Experiments show that VissimRL significantly reduces development effort while maintaining runtime efficiency, and supports consistent improvements in traffic performance during training, as well as emergent coordination in multi-agent control. Overall, VissimRL demonstrates the feasibility of applying RL in high-fidelity simulations and serves as a bridge between academic research and practical applications in intelligent traffic signal control.

Problem

• Traffic Signal Control
• Vissim
• Reinforcement Learning
• Multi-Agent
• Simulation Framework

Innovation

• VissimRL
• multi-agent reinforcement learning
• traffic signal control
• high-fidelity simulation
• Python API