Multi-Agent Reinforcement Learning for Online Traffic Scheduling in Time-Sensitive Application

2026-08-05

AI Summary

This work addresses the challenge of scheduling in Time-Sensitive Networking (TSN) under dynamic multi-XR traffic scenarios in mobile edge computing, where existing methods struggle with complex inter-queue dependencies. To overcome this limitation, the paper proposes a decentralized scheduling framework based on heterogeneous multi-agent reinforcement learning, introducing for the first time the Heterogeneous Agent Proximal Policy Optimization (HAPPO) algorithm to TSN. In this framework, each queue is modeled as an autonomous agent, explicitly capturing inter-queue dependencies and enabling collaborative optimization of scheduling policies. Experimental results demonstrate that the proposed approach reduces average frame queuing delay by up to 26.8% and decreases worst-case latency by approximately 16.8% in dynamic XR environments, significantly enhancing quality of service.

Abstract

Time-sensitive networking (TSN) is increasingly integrated into mobile edge computing (MEC) to support applications with stringent latency requirements, such as extended reality (XR). However, existing TSN scheduling solutions predominantly rely on static optimization techniques or centralized learning models that are based on fixed traffic patterns, limiting their effectiveness in dynamic environments. In practice, MEC environments often host multiple co-located XR traffic flows whose characteristics evolve over time, creating complex inter-queue dependencies that current schedulers fail to capture. Addressing these challenges requires adaptive, decentralized scheduling mechanisms capable of coordinating multiple TSN queues under varying traffic conditions. To this end, this paper proposes a multi-agent reinforcement learning (MARL) framework for TSN scheduling, where each TSN queue is modeled as an autonomous agent. The Heterogeneous-Agent Proximal Policy Optimization (HAPPO) algorithm is employed to explicitly model inter-agent dependencies and jointly optimize service delivery across queues. The simulation results demonstrate that the proposed approach reduces average frame waiting times by up to 26.8% and worst-case delays by approximately 16.8%, highlighting its effectiveness in dynamic XR-driven MEC scenarios.

Problem

• Time-Sensitive Networking
• Mobile Edge Computing
• Multi-Agent Reinforcement Learning
• Traffic Scheduling
• Extended Reality

Innovation

• Multi-Agent Reinforcement Learning
• Time-Sensitive Networking
• Heterogeneous-Agent PPO
• Dynamic Traffic Scheduling
• Mobile Edge Computing