"Two-Stagification": Job Dispatching in Large-Scale Clusters via a Two-Stage Architecture

2025-05-05

AI Summary

To optimize average response time in large-scale FCFS server clusters, this paper proposes a lightweight two-stage architecture: jobs are partitioned based on an adaptive service-time threshold—short jobs are scheduled via JIQ or LWL, while long jobs employ Round-Robin (RR). This design decouples job-size sensitivity from scheduling complexity without requiring real-time server-state awareness, substantially reducing system implementation overhead. Evaluations under Weibull-synthetic workloads and Google cluster traces demonstrate that our approach significantly outperforms single-stage baselines across diverse load conditions and closely approaches the performance of state-of-the-art size-and-state-aware schedulers. The core contribution lies in empirically validating that architectural job partitioning—not scheduler complexity—is a more efficient pathway to near-optimal size-aware scheduling performance.

Technology Category

• Planning, Routing, and Scheduling: Scheduling under Uncertainty
• Machine Learning: Hardware-aware ML
• Search and Optimization: Distributed Search

Application Category

• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

A continuing effort is devoted to devising effective dispatching policies for clusters of First Come First Served servers. Although the optimal solution for dispatchers aware of both job size and server state remains elusive, lower bounds and strong heuristics are known. In this paper, we introduce a two-stage cluster architecture that applies classical Round Robin, Join Idle Queue, and Least Work Left dispatching schemes, coupled with an optimized service-time threshold to separate large jobs from shorter ones. Using both synthetic (Weibull) workloads and real Google data center traces, we demonstrate that our two-stage approach greatly improves upon the corresponding single-stage policies and closely approaches the performance of advanced size- and state-aware methods. Our results highlight that careful architectural design-rather than increased complexity at the dispatcher-can yield significantly better mean response times in large-scale computing environments.

Problem

• Improving job dispatching in large-scale clusters
• Separating large and short jobs effectively
• Reducing mean response times with architectural design

Innovation

• Two-stage cluster architecture for job dispatching
• Combines Round Robin, Join Idle Queue, Least Work Left
• Optimized service-time threshold separates large and small jobs