Problem
Research questions and friction points this paper is trying to address.
Diffusion Models
On-Device Inference
Consumer GPUs
Memory Footprint
Latency
The Weight Is Over - Interactive Diffusion on Consumer GPUs
🤖 AI Summary
研究解决了消费级GPU上扩散模型推理的性能和资源问题，通过嵌入翻译、优化方案和交互式编辑器提高效率和质量。
📝 Abstract
On-device inference is booming, but the momentum is almost all in language models. Diffusion pipelines are memory hungry, latency-sensitive, and require orchestrating an embedder, a transformer, a decoder, and often further postprocessing that is not as standardized as LLM inference loops are. We navigate the trade-off between performance, quality, and model footprint to reach as many client devices in the wild as possible. We make three contributions: an embedding translator that maps a small text encoder into a large encoder space to cut weight and latency; a reproducible sweep recipe for navigating the speed/quality/memory triangle in diffusion pipelines; and an interactive on-device image generation editor achieving sub-second TTFI on recent GPUs.
Innovation
Methods, ideas, or system contributions that make the work stand out.
embedding translator
reproducible sweep recipe
interactive on-device image generation
🔎 Similar Papers
No similar papers found.
