Improving Multiresource Job Scheduling with Markovian Service Rate Policies

2025-04-10

AI Summary

Cloud-based multi-resource job scheduling requires coordinated allocation of heterogeneous resources—such as CPU, memory, and accelerators—to minimize average response time. However, resource coupling and preemption mechanisms render the problem both NP-hard and analytically intractable. This paper introduces the Markov Service Rate (MSR) family of scheduling policies. We theoretically establish that MSR achieves throughput optimality and system stability under three distinct preemption models: full preemption, non-preemption, and preemption with setup overhead. Moreover, we derive the first tight upper bound on mean response time—accurate up to an additive constant—that is both universally applicable and computationally tractable. Leveraging system parameters (job arrival rates, resource demands, and resource capacities), we further design an automated policy selection framework. Extensive experiments demonstrate that MSR significantly outperforms state-of-the-art heuristic schedulers across diverse workloads.

Technology Category

• Planning, Routing, and Scheduling: Scheduling under Uncertainty
• Machine Learning: Hardware-aware ML
• Intelligent Robots: Multi-Robot Systems

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Modern cloud computing workloads are composed of multiresource jobs that require a variety of computational resources in order to run, such as CPU cores, memory, disk space, or hardware accelerators. A single cloud server can typically run many multiresource jobs in parallel, but only if the server has sufficient resources to satisfy the demands of every job. A scheduling policy must therefore select sets of multiresource jobs to run in parallel in order to minimize the mean response time across jobs -- the average time from when a job arrives to the system until it is completed. Unfortunately, achieving low response times by selecting sets of jobs that fully utilize the available server resources has proven to be a difficult problem. In this paper, we develop and analyze a new class of policies for scheduling multiresource jobs, called Markovian Service Rate (MSR) policies. While prior scheduling policies for multiresource jobs are either highly complex to analyze or hard to implement, our MSR policies are simple to implement and are amenable to response time analysis. We show that the class of MSR policies is throughput-optimal in that we can use an MSR policy to stabilize the system whenever it is possible to do so. We also derive bounds on the mean response time under an MSR algorithm that are tight up to an additive constant. These bounds can be applied to systems with different preemption behaviors, such as fully preemptive systems, non-preemptive systems, and systems that allow preemption with setup times. We show how our theoretical results can be used to select a good MSR policy as a function of the system arrival rates, job service requirements, the server's resource capacities, and the resource demands of the jobs.

Problem

• Optimizing multiresource job scheduling to minimize mean response time
• Developing simple, analyzable Markovian Service Rate (MSR) policies
• Ensuring throughput-optimal performance across diverse preemption systems

Innovation

• Uses Markovian Service Rate policies for scheduling
• Simple to implement and analyze response times
• Throughput-optimal and stabilizes system effectively