MagCache: Fast Video Generation with Magnitude-Aware Cache

2025-06-10

AI Summary

Existing video diffusion acceleration methods rely on uniform heuristics or time-embedding variants, requiring extensive prompt-specific calibration—leading to prompt overfitting and output inconsistency. This work identifies a cross-model and cross-prompt universal monotonic decay law governing residual magnitude, and proposes an adaptive amplitude-aware caching mechanism calibrated from only a single sample: it models reconstruction error via residual magnitude ratios to enable dynamic timestep skipping and adaptive feature reuse. Evaluated on Open-Sora and Wan 2.1, our method achieves 2.1× and 2.68× inference speedup, respectively, while consistently outperforming state-of-the-art approaches in LPIPS, SSIM, and PSNR. The core contribution is the first discovery of this universal residual magnitude decay law, enabling a lightweight, highly generalizable, single-sample adaptive acceleration paradigm for video diffusion models.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Existing acceleration techniques for video diffusion models often rely on uniform heuristics or time-embedding variants to skip timesteps and reuse cached features. These approaches typically require extensive calibration with curated prompts and risk inconsistent outputs due to prompt-specific overfitting. In this paper, we introduce a novel and robust discovery: a unified magnitude law observed across different models and prompts. Specifically, the magnitude ratio of successive residual outputs decreases monotonically and steadily in most timesteps while rapidly in the last several steps. Leveraging this insight, we introduce a Magnitude-aware Cache (MagCache) that adaptively skips unimportant timesteps using an error modeling mechanism and adaptive caching strategy. Unlike existing methods requiring dozens of curated samples for calibration, MagCache only requires a single sample for calibration. Experimental results show that MagCache achieves 2.1x and 2.68x speedups on Open-Sora and Wan 2.1, respectively, while preserving superior visual fidelity. It significantly outperforms existing methods in LPIPS, SSIM, and PSNR, under comparable computational budgets.

Problem

• Accelerate video diffusion models without output inconsistency
• Reduce calibration samples needed for adaptive caching
• Maintain visual fidelity while improving generation speed

Innovation

• Magnitude-aware Cache for adaptive timestep skipping
• Error modeling mechanism for robust feature reuse
• Single-sample calibration for efficient video generation