Incast-Free MoE Rate-Based Scheduling

2026-07-28

AI Summary

This work addresses the severe network congestion caused by exponential incast traffic in traditional polling-based scheduling within Mixture-of-Experts (MoE) architectures. For the first time, this study identifies and characterizes this incast problem and proposes MoE-aware proactive fair-rate scheduling: a framework that models traffic distributions to predict hotspots and integrates rate-based fair scheduling at the NIC level to eliminate incast entirely. Experimental results demonstrate that the proposed approach completely eradicates incast under both real-world and synthetic workloads, achieves near-optimal link utilization approaching 100%, significantly reduces collective communication time (CCT), and remains practical for real-world deployment.

Abstract

Mixture of Experts (MoE) architectures have become key to large language models; however, their typical round-robin (RR) scheduling introduces significant bottlenecks. In this paper, we demonstrate that RR causes a previously-undiscovered exponential incast phenomenon with MoE traffic. We propose an alternative proactive fair scheduling framework tailored for MoE workloads, which effectively prevents fabric oversubscription. We also outline how it can be implemented in NICs. Finally, through extensive simulations with real and synthetic workloads, we demonstrate that this framework consistently eliminates incast, maintains a near-100% link utilization, and reduces Collective Completion Time (CCT).

Problem

• Mixture of Experts
• incast
• scheduling
• network congestion
• large language models

Innovation

• Mixture of Experts
• incast
• rate-based scheduling
• fair scheduling
• fabric oversubscription