Adaptive Approach to Enhance Machine Learning Scheduling Algorithms During Runtime Using Reinforcement Learning in Metascheduling Applications

2025-09-24

AI Summary

Traditional offline construction of Multi-Schedule Graphs (MSGs) fails to accommodate dynamic scenarios such as hardware failures and runtime slack variations, resulting in insufficient robustness of AI-driven scheduling in time-triggered systems. This paper proposes an online adaptive scheduling method based on reinforcement learning (RL), embedding an RL agent within a meta-scheduler to enable real-time MSG expansion, continuous scheduling policy optimization, and support for context-aware adaptation, mode switching, and dynamic performance-bound adjustment. Our key contribution lies in overcoming the limitations of offline training by enabling incremental, runtime construction of the MSG and concurrent exploration of scheduling policies. Experimental evaluation under strict deadline constraints demonstrates a 23.6% improvement in scheduling success rate and enhanced fault recovery capability; moreover, response latency to timing fluctuations and unexpected events is reduced by 41%, significantly improving system reliability and adaptability.

Technology Category

• Planning, Routing, and Scheduling: Planning/Scheduling and Learning
• Multiagent Systems: Multiagent Learning
• Machine Learning: Online Learning & Bandits

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Metascheduling in time-triggered architectures has been crucial in adapting to dynamic and unpredictable environments, ensuring the reliability and efficiency of task execution. However, traditional approaches face significant challenges when training Artificial Intelligence (AI) scheduling inferences offline, particularly due to the complexities involved in constructing a comprehensive Multi-Schedule Graph (MSG) that accounts for all possible scenarios. The process of generating an MSG that captures the vast probability space, especially when considering context events like hardware failures, slack variations, or mode changes, is resource-intensive and often infeasible. To address these challenges, we propose an adaptive online learning unit integrated within the metascheduler to enhance performance in real-time. The primary motivation for developing this unit stems from the limitations of offline training, where the MSG created is inherently a subset of the complete space, focusing only on the most probable and critical context events. In the online mode, Reinforcement Learning (RL) plays a pivotal role by continuously exploring and discovering new scheduling solutions, thus expanding the MSG and enhancing system performance over time. This dynamic adaptation allows the system to handle unexpected events and complex scheduling scenarios more effectively. Several RL models were implemented within the online learning unit, each designed to address specific challenges in scheduling. These models not only facilitate the discovery of new solutions but also optimize existing schedulers, particularly when stricter deadlines or new performance criteria are introduced. By continuously refining the AI inferences through real-time training, the system remains flexible and capable of meeting evolving demands, thus ensuring robustness and efficiency in large-scale, safety-critical environments.

Problem

• Overcoming offline AI scheduling limitations in dynamic environments
• Addressing resource-intensive Multi-Schedule Graph generation complexity
• Enhancing real-time adaptation to unexpected events and deadlines

Innovation

• Online reinforcement learning unit for metascheduler adaptation
• Dynamic expansion of Multi-Schedule Graph during runtime
• Real-time AI inference refinement for scheduling optimization