Improving Offline Mixed-Criticality Scheduling with Reinforcement Learning

2025-04-04

AI Summary

This paper addresses the NP-hard problem of offline, non-preemptive mixed-criticality (MC) real-time scheduling on heterogeneous multicore platforms. We propose the first systematic reinforcement learning (RL)-based solution, modeling scheduling as a Markov decision process and employing PPO and DQN algorithms to jointly maximize overall task completion rate while guaranteeing schedulability of high-criticality tasks. Our approach innovatively overcomes the performance limitations of conventional static analysis and heuristic methods, enabling adaptive handling of dynamic workloads and processor-speed heterogeneity. Evaluated on 100,000 synthetic instances and real-world datasets, our method achieves an average task completion rate of 80% (85% for high-criticality tasks), rising to 94% (93% for high-criticality tasks) under stable scenarios—significantly outperforming state-of-the-art offline MC schedulers.

Technology Category

Planning, Routing, and Scheduling: Learning for Planning and SchedulingMachine Learning: Reinforcement LearningMultiagent Systems: Multiagent Learning

Application Category

Economics, Online Markets and Human Computation: Cost models of using LLMs in production systemsSystems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applicationsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper introduces a novel reinforcement learning (RL) approach to scheduling mixed-criticality (MC) systems on processors with varying speeds. Building upon the foundation laid by [1], we extend their work to address the non-preemptive scheduling problem, which is known to be NP-hard. By modeling this scheduling challenge as a Markov Decision Process (MDP), we develop an RL agent capable of generating near-optimal schedules for real-time MC systems. Our RL-based scheduler prioritizes high-critical tasks while maintaining overall system performance. Through extensive experiments, we demonstrate the scalability and effectiveness of our approach. The RL scheduler significantly improves task completion rates, achieving around 80% overall and 85% for high-criticality tasks across 100,000 instances of synthetic data and real data under varying system conditions. Moreover, under stable conditions without degradation, the scheduler achieves 94% overall task completion and 93% for high-criticality tasks. These results highlight the potential of RL-based schedulers in real-time and safety-critical applications, offering substantial improvements in handling complex and dynamic scheduling scenarios.

Problem

• Solves non-preemptive NP-hard scheduling in mixed-criticality systems
• Uses RL to prioritize high-critical tasks efficiently
• Improves task completion rates in dynamic real-time systems

Innovation

• Reinforcement learning for mixed-criticality scheduling
• Non-preemptive scheduling via Markov Decision Process
• RL agent ensures high-critical task prioritization