ExpertPlex: A High-Goodput Disaggregated Serving System for MoE LLMs with Adaptive Persistent Kernels

2026-07-20

AI Summary

Efficient deployment of large-scale Mixture-of-Experts (MoE) models is hindered by their massive parameter count and dynamic computation patterns, which lead to coarse-grained resource allocation, inter-stage load imbalance, and network interference. This work proposes an expert-sharing and attention-decoupled architecture that reuses expert modules across pipeline stages and isolates lightweight attention computations. Coupled with a tile-granularity adaptive persistence kernel and an attention-triggered communication scheduling mechanism, the design significantly enhances compute-communication overlap and eliminates network contention. Evaluated on MiniMax-M2.7 and GLM-5.1-FP8, the system achieves up to 2.01× higher goodput than instance-level decoupling and 1.66× improvement over colocated baselines.

Abstract

LLMs scale Mixture-of-Experts (MoE) parameters for superior intelligence, but massive weights and dynamic computation impede efficient serving. Existing instance-level prefill-decode disaggregation isolates the phases on separate full-model replicas. As MoE weights grow, each instance may span tens to hundreds of GPUs, making resource allocation increasingly coarse. Configured prefill-to-decode ratios thus often mismatch demand, overprovisioning one phase while overloading the other. Prefill-decode colocation avoids this duplication, but existing Green Context solutions partition each GPU by phase and fix phase resources during a kernel. They cannot track resource changes across operations or layerwise variation in routed expert load, causing head-of-line blocking or idle reserved resources. Partitioning every GPU also leaves each phase with fewer local resources, forces wider parallelism and more communication, and lets prefill and decode traffic interfere on the shared network. We present ExpertPlex, which shares massive MoE experts across phases while disaggregating lightweight attention modules. Expert sharing eliminates over 95\% duplicate model weights and multiplexes dynamically sparse computation, while attention disaggregation reduces attention communication cost. ExpertPlex further uses (1) adaptive persistent kernels to schedule dynamic expert computation at tile granularity for efficient, isolated execution; (2) attention-initiated MoE communication to avoid network interference and enable cross-phase communication-computation overlap; and (3) a tile-to-cluster model to optimize these mechanisms for maximum goodput. Experiments serving MiniMax-M2.7 and GLM-5.1-FP8 show that ExpertPlex improves goodput by up to 2.01$\times$ over instance-level prefill-decode disaggregation and 1.66$\times$ over prefill-decode colocation.

Problem

• Mixture-of-Experts
• LLM serving
• prefill-decode disaggregation
• resource allocation
• dynamic computation

Innovation

• Mixture-of-Experts
• Disaggregated Serving
• Adaptive Persistent Kernels
• Goodput Optimization
• Attention-MoE Decoupling