TetriServe: Efficient DiT Serving for Heterogeneous Image Generation

2025-10-01

AI Summary

Existing DiT models incur substantial computational overhead in high-resolution image generation, while static parallelism strategies fail to accommodate heterogeneous workloads featuring mixed resolutions and diverse deadline requirements—leading to low GPU utilization and insufficient SLO compliance. This paper proposes a fine-grained parallel scheduling framework that, for the first time, applies sequence parallelism at the denoising-step level. We design a deadline-aware scheduler based on a timing-wheel structure and a request co-batching mechanism, enabling dynamic alignment between workload characteristics and resource provisioning. Leveraging the staged nature of DiT’s iterative denoising process, our approach significantly improves service efficiency without compromising image quality. Experiments on mainstream DiT models demonstrate up to a 32% improvement in SLO attainment rate, substantial reduction in per-image GPU latency, and markedly higher GPU utilization.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Diffusion Transformer (DiT) models excel at generating highquality images through iterative denoising steps, but serving them under strict Service Level Objectives (SLOs) is challenging due to their high computational cost, particularly at large resolutions. Existing serving systems use fixed degree sequence parallelism, which is inefficient for heterogeneous workloads with mixed resolutions and deadlines, leading to poor GPU utilization and low SLO attainment. In this paper, we propose step-level sequence parallelism to dynamically adjust the parallel degree of individual requests according to their deadlines. We present TetriServe, a DiT serving system that implements this strategy for highly efficient image generation. Specifically, TetriServe introduces a novel round-based scheduling mechanism that improves SLO attainment: (1) discretizing time into fixed rounds to make deadline-aware scheduling tractable, (2) adapting parallelism at the step level and minimize GPU hour consumption, and (3) jointly packing requests to minimize late completions. Extensive evaluation on state-of-the-art DiT models shows that TetriServe achieves up to 32% higher SLO attainment compared to existing solutions without degrading image quality.

Problem

• Optimizing DiT serving efficiency for heterogeneous image generation
• Addressing poor GPU utilization in mixed-resolution workloads
• Improving SLO attainment through dynamic step-level parallelism

Innovation

• Step-level sequence parallelism for dynamic adjustment
• Round-based scheduling mechanism for deadline awareness
• Joint request packing to minimize late completions