Frequency-Aware Error-Bounded Caching for Accelerating Diffusion Transformers

2026-03-05

AI Summary

This work addresses the high computational cost of iterative denoising in Diffusion Transformers (DiT), where existing caching methods fail to account for the non-uniformity of the denoising process across timesteps, network depth, and feature dimensions, thereby compromising the trade-off between efficiency and fidelity. To this end, we propose SpectralCache, a plug-and-play, training-free caching framework that, for the first time, systematically characterizes the multi-dimensional non-uniformity in DiT denoising and introduces an error-bounded caching mechanism. Its core components include Timestep-Aware Dynamic Scheduling (TADS), Cumulative Error Budgeting (CEB), and Frequency-Domain Decomposition Caching (FDC). Evaluated on FLUX.1-schnell, SpectralCache achieves a 2.46× speedup with LPIPS of 0.217 and SSIM of 0.727, outperforming TeaCache by 16% in speed while maintaining image quality within 1% relative difference.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Matrix & Tensor Methods
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Diffusion Transformers (DiTs) have emerged as the dominant architecture for high-quality image and video generation, yet their iterative denoising process incurs substantial computational cost during inference. Existing caching methods accelerate DiTs by reusing intermediate computations across timesteps, but they share a common limitation: treating the denoising process as uniform across time,depth, and feature dimensions. In this work, we identify three orthogonal axes of non-uniformity in DiT denoising: (1) temporal -- sensitivity to caching errors varies dramatically across the denoising trajectory; (2) depth -- consecutive caching decisions lead to cascading approximation errors; and (3) feature -- different components of the hidden state exhibit heterogeneous temporal dynamics. Based on these observations, we propose SpectralCache, a unified caching framework comprising Timestep-Aware Dynamic Scheduling (TADS), Cumulative Error Budgets (CEB), and Frequency-Decomposed Caching (FDC). On FLUX.1-schnell at 512x512 resolution, SpectralCache achieves 2.46x speedup with LPIPS 0.217 and SSIM 0.727, outperforming TeaCache (2.12x, LPIPS 0.215, SSIM 0.734) by 16% in speed while maintaining comparable quality (LPIPS difference<1%). Our approach is training-free, plug-and-play, and compatible with existing DiT architectures.

Problem

• Diffusion Transformers
• caching
• non-uniformity
• inference acceleration
• error-bounded

Innovation

• Frequency-Aware Caching
• Diffusion Transformers
• Error-Bounded Acceleration
• Temporal Non-Uniformity
• Spectral Decomposition