The Golden Path Hypothesis: Reusable Schedules in Diffusion Caching

2026-09-30

AI Summary

This study addresses the poor generalizability of cache scheduling in diffusion models caused by its dependence on specific prompts. We propose the "golden path" hypothesis, which reveals the structural similarity of denoising trajectories and error accumulation patterns under fixed inference conditions. Building upon this insight, we establish an end-to-end, universal cache scheduling method through error decomposition and large-scale exhaustive search. Our findings demonstrate that only a small number of samples suffice to identify optimal scheduling schemes that generalize across datasets. The proposed approach significantly accelerates generation while preserving high-quality outputs, thereby achieving universally efficient inference without requiring prompt-specific customization.

Abstract

Diffusion caching accelerates generation by replacing transformer computation with cached or predicted features at selected denoising steps. We introduce the Golden Path Hypothesis (GPH): under fixed inference conditions, prompt-independent cache schedules can achieve final-output quality comparable to the best prompt-specific schedules across prompts. We investigate the GPH across ten caching methods, four image and video models, and three cache ratios. Prompt-adaptive methods repeatedly select a small number of schedules, and reusing their most frequent schedules on new prompts closely matches the quality of prompt-specific choices. Exhaustive evaluation of 1.4 million schedules on four examples further identifies prompt-independent schedules that remain competitive on unseen prompts. To explain this transfer, we analyze denoising trajectories and the accumulation of caching errors. Latent-state trajectories exhibit similar structures across datasets and seeds, while an exact error decomposition shows that accumulated effects of earlier errors predict final latent-state error better than local approximation errors. This motivates searching for end-to-end schedules using final-output quality. With only a small set of examples, the resulting golden paths transfer across prompts and datasets, and can be tuned to the desired quality objective, including reconstruction fidelity or perceptual similarity.

Problem

• Diffusion Caching
• Cache Schedules
• Denoising Steps
• Generation Acceleration
• Golden Path Hypothesis

Innovation

• Diffusion Caching
• Golden Path Hypothesis
• Cache Schedules
• Error Accumulation
• Denoising Trajectories