ServerlessT2I: Efficient Text-to-Image Workflow Serving on a Serverless Platform

2026-07-29

AI Summary

This work addresses the inefficiencies of current serverless platforms that deploy text-to-image workflows as monolithic black-box GPU functions, resulting in high scaling overhead, manual GPU coordination, and insufficient fine-grained fairness. To overcome these limitations, we propose a fine-grained workflow decomposition mechanism tailored for serverless architectures, which decomposes workflows into loosely coupled model functions enabling independent scaling and declarative orchestration. We further introduce an efficient data plane by reclaiming idle GPU memory to support transparent inter-function communication and low-overhead model loading. Additionally, a fairness-aware scheduler is designed to ensure quality-of-service guarantees for multi-tenant workloads. Experimental results demonstrate that our approach achieves up to 2× higher throughput under the same GPU budget, reduces GPU resource consumption by up to 3× under fixed workloads, and consistently meets service-level objectives.

Abstract

Text-to-image (T2I) workflows are increasingly deployed on serverless platforms because users often compose customized workflows and invoke them intermittently. Existing platforms typically deploy each workflow as an opaque GPU function, provisioning, placing, and scaling all constituent models in the workflow together. This monolithic design obscures workflow structure, inflates scaling overhead, forces users to manage low-level GPU coordination, and limits fine-grained fairness in multi-tenant clusters. In this paper, we present ServerlessT2I, a serverless-native system that decomposes a T2I workflow into loosely coupled model functions that can be independently managed and scheduled. By explicitly managing individual model execution, ServerlessT2I enables per-model scaling, declarative workflow composition, transparent GPU-resident communication, and fairness-aware scheduling. To make this decomposition efficient, ServerlessT2I harvests slack GPU memory left idle by compute-bound T2I inference to build a data plane that reduces model loading and data communication overheads. \sys{} further introduces a fair scheduler for multi-tenant serving. Using production traces, ServerlessT2I sustains up to 2$\times$ higher request rates than existing T2I workflow serving systems with the same GPU budget; for a fixed request rate, it saves up to 3$\times$ GPU resources while satisfying service level objectives (SLOs).

Problem

• serverless
• text-to-image
• workflow serving
• GPU resource management
• multi-tenant fairness

Innovation

• serverless
• text-to-image
• workflow decomposition
• GPU memory reuse
• fair scheduling