Problem
Research questions and friction points this paper is trying to address.
Truncated Diffusion Sampler
Power-Law Spectra
Spectral Truncation
Sampling Complexity
How Many Directions Must a Truncated Diffusion Sampler Retain? Matching Bounds Under Power-Law Spectra
🤖 AI Summary
This study investigates the number of directions that truncated diffusion samplers must retain when operating on power-law spectral data. By integrating random matrix theory, Wiener filtering analysis, and convergence bounds for diffusion models, we derive matching theoretical limits under Gaussian data assumptions. We demonstrate that the truncation error is governed by the cumulative Wiener gains of the omitted directions, revealing a non-vanishing error phenomenon arising from the aggregation of weak-signal components. Building upon these insights, we propose a subspace selection criterion based on spectral tail error budgets, along with an upper bound on sampling step complexity. Furthermore, we validate that this framework extends effectively to principal component estimation and mixture distributions under exact score conditions.
📝 Abstract
Diffusion samplers can reduce computation by generating selected spectral coordinates and filling the remaining directions with noise. How many directions must they retain? We study this question for data with power-law covariance spectra. For Gaussian data compared to a smoothed target, we prove matching bounds on the required number of retained directions, provided that the ambient dimension is sufficiently large. The truncation error depends on the combined Wiener gains of the omitted directions, regardless of the accuracy of the sampler on the retained coordinates. Keeping only directions whose signal exceeds the output noise level can therefore leave a non-vanishing error: many individually weak directions remain significant in aggregate. Combining this characterization with a diffusion convergence bound yields sufficient sampling-step complexity under exact scores. The upper bounds also extend to estimated principal components and, componentwise, to Gaussian mixtures. The practical prescription is to select the retained subspace using an aggregate spectral-tail error budget, then to choose the diffusion noise level accordingly.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Truncated Diffusion Sampler
Power-Law Spectra
Matching Bounds
Wiener Gains
Spectral-Tail Error Budget
🔎 Similar Papers
No similar papers found.
