Analytical Power-Aware Provisioning for Prefill-Decode Disaggregated AI Inference

2026-09-21

AI Summary

本文针对AI推理服务中的功耗与服务能力权衡问题，提出了一种分析框架，用于在考虑硬件限制和工作负载特征的情况下优化预填充-解码实例的数量。

Abstract

Power availability increasingly constrains the operation of AI inference fleets, creating a need for provisioning methods that jointly consider serving capacity and power consumption. Prefill--decode (PD) disaggregation has emerged as a prevalent architecture for large-scale inference serving. However, determining the appropriate numbers of prefill and decode instances is challenging because serving capacity depends jointly on workload characteristics, hardware constraints, queueing, and KV-cache reservations. Existing approaches largely rely on profiling and simulation, providing limited analytical insight into how provisioning decisions shape the tradeoff between serving capacity and power consumption. This paper develops an analytical framework for power-aware provisioning of PD-disaggregated AI inference. Given an inference workload and hardware, the framework models the serving capacity and average power consumption of a provisioned deployment. The serving-capacity model is derived from the joint distribution of input--output lengths and hardware compute and memory limits. In particular, it explicitly captures the coupling between prefill and decode induced by KV-cache reservations, as well as the impact of request queueing. On this basis, the power model determines per-instance power consumption as a function of normalized serving throughput. Together, the models determine the serving capacity--power Pareto front among candidate provisioned deployments, enabling the service provider to choose a provisioned deployment as the workload or available power changes.

Problem

• Power-aware provisioning
• Prefill-decode disaggregation
• Serving capacity
• Power consumption
• KV-cache

Innovation

• power-aware provisioning
• prefill-decode disaggregation
• analytical framework
• serving capacity
• KV-cache