When Does Disaggregation Pay? Simulating Prefill--Decode--Attention--FFN Specialization for Agentic LLM Inference

2026-08-04

AI Summary

This work addresses the challenge that complex workloads in large language model (LLM) reasoning for intelligent agents cannot be efficiently supported by conventional homogeneous GPUs. To this end, the paper introduces HeteroPanacea—the first three-dimensional framework enabling cross-stack simulation—systematically evaluating specialized deployment of the four-stage Prefill-Decoding-Attention-FFN (PDAF) pipeline across heterogeneous hardware. The framework incorporates decoupled quantization, automated intra- and inter-device parallel scheduling, and a PDAF-tailored NPU architecture. Large-scale experiments demonstrate that decoupling Prefill and Decode stages improves throughput by up to 75%, while four-way PDAF decomposition consistently enhances inference efficiency across diverse models, revealing an underlying relationship between model structure and the benefits of workload decoupling.

Abstract

Agentic inference now dominates the LLM inference landscape, requiring LLMs to actively engage in multi-turn interactions with tool-calling capabilities. This introduces a more complex workload for the underlying inference system: serving stages such as prefill and decode exhibit substantially different behaviors and demand distinct compute and memory-bandwidth capabilities. As a result, a single homogeneous GPU system now struggles to support agentic inference, motivating an industry shift toward heterogeneous systems with disaggregated serving capabilities, such as the emerging Vera-Rubin platform with GPUs and Groq LPUs. However, the question of what the optimal hardware should look like for each component in a heterogeneous system remains underexplored. To this end, we propose a novel simulation framework for disaggregated serving, termed \textbf{HeteroPanacea}, that enables system-level simulation across three dimensions: 1) disaggregated quantization, 2) automated intra- and inter-device parallelization scheduling, and 3) PDAF (prefill-decode-attention-FFN) NPU architectural heterogeneity. By combining these three axes, we provide a cross-stack simulation framework for future heterogeneous agentic serving systems. We confirm the benefit of Prefill Decode disaggregation, simulating increased serving throughput by up to 75\% compared to traditional serving with current GPUs and demonstrate 4 way Prefill Decode Attention FFN disaggregation is the most consistent for increasing throughput across different models, assuming custom NPUs. We also investigate the relationship between model architecture and gain from disaggregation by running a set of ablation studies.

Problem

• agentic inference
• disaggregation
• heterogeneous systems
• LLM inference
• hardware specialization

Innovation

• disaggregated serving
• heterogeneous inference
• PDAF specialization
• HeteroPanacea
• agentic LLM