🤖 AI Summary
Existing diffusion models fully degrade data into white noise, resulting in challenging denoising tasks, high computational overhead, and severe loss of spectral structure. To address this, we propose InSPECT—a diffusion model that explicitly preserves invariant spectral features throughout both forward and reverse processes. InSPECT is the first to systematically model and constrain Fourier-domain dynamics, enabling spectral coefficients to smoothly converge toward controllable stochastic noise while jointly preserving structural fidelity, generation diversity, and randomness. Our approach comprises a spectral-aware forward process and a dedicated Fourier-domain denoising network. Evaluated on CIFAR-10, CelebA, and LSUN, InSPECT achieves a 39.23% reduction in FID and a 45.80% improvement in Inception Score (IS) over DDPM, with accelerated convergence and significantly smoother diffusion trajectories.
📝 Abstract
Modern diffusion models (DMs) have achieved state-of-the-art image generation. However, the fundamental design choice of diffusing data all the way to white noise and then reconstructing it leads to an extremely difficult and computationally intractable prediction task. To overcome this limitation, we propose InSPECT (Invariant Spectral Feature-Preserving Diffusion Model), a novel diffusion model that keeps invariant spectral features during both the forward and backward processes. At the end of the forward process, the Fourier coefficients smoothly converge to a specified random noise, enabling features preservation while maintaining diversity and randomness. By preserving invariant features, InSPECT demonstrates enhanced visual diversity, faster convergence rate, and a smoother diffusion process. Experiments on CIFAR-10, Celeb-A, and LSUN demonstrate that InSPECT achieves on average a 39.23% reduction in FID and 45.80% improvement in IS against DDPM for 10K iterations under specified parameter settings, which demonstrates the significant advantages of preserving invariant features: achieving superior generation quality and diversity, while enhancing computational efficiency and enabling faster convergence rate. To the best of our knowledge, this is the first attempt to analyze and preserve invariant spectral features in diffusion models.