score-guided dataset distillation

This paper addresses the lack of a unified theoretical foundation for guidance mechanisms in diffusion models. We propose the first integrated algorithmic-theoretic framework unifying classifier-guided diffusion (CFG) and reward-guided diffusion. Methodologically, we introduce a reward-gradient term based on score differences into the reverse diffusion process, enabling efficient reward guidance without requiring full-trajectory reward modeling or training. Theoretically, we establish that CFG inherently optimizes the expected reciprocal of classifier probabilities and develop novel analytical tools—score-difference analysis and an upper bound estimator for expected reward—under our framework. Our contributions are fourfold: (1) a unified theoretical interpretation of both guidance paradigms; (2) a new sampling paradigm eliminating the need for trajectory-based reward training; (3) rigorous theoretical guarantees that our guidance strategy strictly improves reward metrics; and (4) empirical validation demonstrating superior generation quality and training stability across diverse benchmarks.

This work addresses the lack of a systematic theoretical understanding of how finite-sample learning, neural network parameterization, and numerical discretization jointly affect generation quality in diffusion models. The authors develop a unified framework for convergence and generalization analysis, decomposing the overall generation error— for the first time—into four interpretable components: forward truncation error, backward discretization error, generalization error (accounting for both data finiteness and forward discretization), and optimization gap. Leveraging a ResNet-type score estimator and combining tools from numerical analysis of stochastic differential equations with total variation distance bounds, they quantitatively characterize the joint influence of training sample size, temporal grid density, and optimization accuracy on generation fidelity, thereby establishing end-to-end theoretical guarantees.

This work addresses the issue of distribution drift and performance degradation that arises when diffusion models are recursively trained on synthetic data. It presents the first rigorous theoretical framework analyzing the distributional shift induced by such recursive training, deriving tight upper and lower bounds on the cumulative discrepancy between the generated and target distributions. The analysis quantitatively links this discrepancy to the score estimation error and the proportion of fresh real data incorporated during training. Through score function modeling, divergence analysis, and empirical validation, the study characterizes distinct drift regimes under different training mechanisms and demonstrates the accuracy of its theoretical predictions on both synthetic and image datasets.

Standard discriminator guidance employs cross-entropy loss, yet theoretical analysis reveals it increases the KL divergence between the model and data distributions—especially under discriminator overfitting—degrading generation quality. Method: We propose the first theoretically guaranteed discriminator guidance framework converging to the true data distribution: we reformulate the discriminator objective to directly minimize KL divergence, ensuring that Score-Matching diffusion models are guided toward the target distribution with strict theoretical guarantees. Contribution/Results: Through rigorous convergence analysis and extensive experiments across multiple datasets, our method consistently suppresses distributional shift, improving FID by 12–28% across all benchmarks. It significantly enhances sample fidelity and training robustness, establishing a principled foundation for reliable, high-quality diffusion-based generation.

This work addresses the challenge of zero-shot conditional generation from pretrained unconditional diffusion models—specifically, generating samples satisfying complex logical constraints (e.g., structural conditions on tables, images, or time series) without fine-tuning. We propose a neural-symbolic soft-constraint embedding method that encodes first-order logic constraints as differentiable soft penalties and directly perturbs the score function to achieve theoretically consistent approximation of the conditional distribution—bypassing classifier-guided sampling or costly retraining. Our approach integrates score-based modeling, symbolic logic encoding, score correction, and stabilized sampling. Experiments across diverse data modalities demonstrate that our method achieves high-fidelity approximation of the true conditional distribution, significantly outperforming existing zero-shot conditional generation baselines.

This work addresses the challenges of sampling and fine-tuning diffusion and flow models under exponentially tilted target distributions by proposing a unified framework grounded in stochastic optimal control and nonequilibrium thermodynamics. The framework integrates adjoint matching with a novel score-matching approach, introduces a bias-variance decomposition to characterize gradient variance properties, and establishes norm bounds for the adjoint ordinary differential equation. It further extends CMCD/NETS losses and the Crooks/Jarzynski identities to the exponentially tilted setting. Experiments on Stable Diffusion 1.5 and 3 demonstrate the efficacy of the proposed method, showing that adjoint-based approaches not only exhibit finite gradient variance but also significantly outperform baseline methods in reward fine-tuning and unnormalized density sampling tasks.

Existing work lacks a theoretical explanation for why diffusion guidance consistently yields high-quality samples, particularly regarding how it ensures generated samples remain within the support of the target distribution. This work introduces the notion of “support robustness” and, under the assumption of an exact score function, rigorously proves for the first time that diffusion guidance almost surely produces samples arbitrarily close to the target support, thereby guaranteeing structural plausibility and physical meaningfulness. The analysis unifies a broad class of discretization schemes—including DDPM, DDIM, and those based on exponential integrators—providing a solid theoretical foundation for the empirical success of diffusion guidance in generating high-fidelity samples.

Existing diffusion models often suffer from low constraint satisfaction rates or degraded sample quality when generating samples under complex feasibility constraints, primarily due to distributional mismatches between training and sampling phases. This work proposes a trajectory-aware fine-tuning framework based on online rollouts, which integrates constraint guidance during training and, for the first time, incorporates a numerical integration perspective into the diffusion process. By end-to-end differentiating denoising trajectories under a fixed noise schedule, the method explicitly exposes constraint violations, thereby aligning the training and sampling distributions. Combining constraint-aware guidance, differentiable noise scheduling, and an online rollout mechanism, the approach significantly improves constraint satisfaction across multiple tasks while maintaining generation quality on par with current state-of-the-art methods.

This study addresses the issue that single-evaluation paradigms obscure distribution coverage loss in diffusion distillation models. We introduce the Pass@k metric to diffusion model evaluation for the first time, systematically analyzing generation diversity under multi-step sampling by integrating classifier-free guidance (CFG) comparisons with consistency- and trajectory-guided distillation techniques. Our findings reveal that training objectives determine whether student models inherit or sacrifice the teacher’s distribution coverage. Specifically, high CFG strength improves quality under small sampling budgets yet degrades coverage under large ones. Furthermore, distribution-matching objectives concentrate outputs, whereas consistency objectives better preserve coverage properties. This work corrects the prevailing misconception that distilled models invariably outperform their teachers, explicitly elucidating the quality-diversity trade-off mechanisms in image and video generation.

This work addresses the longstanding challenge in diffusion modeling of simultaneously enabling simulation-free training and finite-time generation. The authors propose a novel reference diffusion process whose marginal distributions exactly match the target distribution, and whose time-varying conditional distributions facilitate a well-defined reversal. This formulation reveals that score matching naturally arises as the consequence of reversing the reference process and further shows that conditional flow matching corresponds to its small-noise limiting case. The resulting framework is the first to jointly support training without requiring forward simulations and generation within a finite time horizon, thereby not only broadening the theoretical foundations of diffusion models but also enhancing their practical flexibility.