Multi-residual Mixture of Experts Learning for Cooperative Control in Multi-vehicle Systems

2025-07-13

AI Summary

To address the poor generalizability of autonomous vehicles (AVs) as mobile actuators in Lagrangian traffic control—particularly under mixed traffic flows, conflicting multi-agent objectives, and physical constraints—this paper proposes a residual-enhanced cooperative control framework. The framework integrates residual reinforcement learning with a scene-aware mixture-of-experts mechanism, explicitly modeling traffic topology while dynamically selecting and refining nominal control policies to enhance adaptability and robustness in multi-agent coordination. Trained and validated via data-driven simulation using real-world intersection datasets from multiple cities, the approach demonstrates significant improvements in eco-driving performance: across empirical deployments in Atlanta, Dallas–Fort Worth, and Salt Lake City, it achieves a 4%–9% reduction in aggregate vehicle emissions compared to the best-performing baseline.

Technology Category

• Multiagent Systems: Multiagent Learning
• Planning, Routing, and Scheduling: Planning with Language Models
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Autonomous vehicles (AVs) are becoming increasingly popular, with their applications now extending beyond just a mode of transportation to serving as mobile actuators of a traffic flow to control flow dynamics. This contrasts with traditional fixed-location actuators, such as traffic signals, and is referred to as Lagrangian traffic control. However, designing effective Lagrangian traffic control policies for AVs that generalize across traffic scenarios introduces a major challenge. Real-world traffic environments are highly diverse, and developing policies that perform robustly across such diverse traffic scenarios is challenging. It is further compounded by the joint complexity of the multi-agent nature of traffic systems, mixed motives among participants, and conflicting optimization objectives subject to strict physical and external constraints. To address these challenges, we introduce Multi-Residual Mixture of Expert Learning (MRMEL), a novel framework for Lagrangian traffic control that augments a given suboptimal nominal policy with a learned residual while explicitly accounting for the structure of the traffic scenario space. In particular, taking inspiration from residual reinforcement learning, MRMEL augments a suboptimal nominal AV control policy by learning a residual correction, but at the same time dynamically selects the most suitable nominal policy from a pool of nominal policies conditioned on the traffic scenarios and modeled as a mixture of experts. We validate MRMEL using a case study in cooperative eco-driving at signalized intersections in Atlanta, Dallas Fort Worth, and Salt Lake City, with real-world data-driven traffic scenarios. The results show that MRMEL consistently yields superior performance-achieving an additional 4%-9% reduction in aggregate vehicle emissions relative to the strongest baseline in each setting.

Problem

• Designing effective Lagrangian traffic control policies for AVs
• Handling diverse real-world traffic environments robustly
• Addressing multi-agent complexity and conflicting optimization objectives

Innovation

• Multi-Residual Mixture of Expert Learning framework
• Dynamic selection of nominal policies
• Residual reinforcement learning for corrections