score-matching denoiser training

This work addresses the lack of theoretical guarantees for plug-and-play (PnP) methods in ill-posed linear inverse problems when the denoiser is not consistent with the physical forward model. The authors propose a PnP forward–backward splitting algorithm based on the minimum mean square error (MMSE) estimator, incorporating a linear operator matched to the covariance structure of the observation noise, and extend it to neural network–parameterized denoisers. They establish, for the first time, that denoisers cannot be designed independently of the forward model. Under mild assumptions, they provide recovery guarantees—both pointwise and in Wasserstein distance—for MMSE and neural network denoisers in the presence of Gaussian noise with non-diagonal covariance, and rigorously prove the convergence and reconstruction performance of the proposed algorithm.

This work addresses the implicit regularization structure and convergence properties of plug-and-play (PnP) methods employing minimum mean squared error (MMSE) denoisers. We establish, for the first time, that MMSE-PnP is equivalent to explicit optimization with a 1-weakly convex regularizer given by the upper Moreau envelope of the negative log marginal density. Building on this insight, we derive the first non-asymptotic sublinear convergence rate for PnP gradient descent—removing restrictive asymptotic or strong convexity assumptions required in prior analyses. Our approach integrates weak convexity theory, proximal operator analysis, and implicit regularization modeling. We validate the theoretical convergence behavior and practical efficacy through synthetic 1D experiments and real-world inverse problems, including image deblurring and CT reconstruction. Key contributions include: (i) characterizing the precise implicit regularizer induced by MMSE denoisers; (ii) providing the first non-asymptotic convergence guarantee for PnP gradient descent; and (iii) demonstrating tight alignment between theory and empirical performance.

This work identifies an inherent bias in diffusion models when optimizing conditional inputs via denoising score matching: the optimization breaks theoretical equivalence with exact score matching and systematically inflates the norm of the score function. Method: Leveraging score matching theory and the diffusion framework, the authors derive a rigorous mathematical characterization and quantitative model of this bias, proving it arises from perturbations to score estimation induced by input optimization—and showing that analogous bias occurs even under pre-trained model-based data distribution optimization. Contribution/Results: The analysis establishes the bias’s cross-method universality, confirming its presence in prominent approaches including MAR, PerCo, and DreamFusion. Beyond delivering a critical theoretical warning for existing methods, the work provides foundational insights for designing optimization strategies that mitigate score-norm inflation—advancing both the theoretical understanding and practical design of conditional diffusion modeling.

This work addresses the challenges of manifold mismatch and lack of convergence guarantees when directly embedding score-based generative models into optimization algorithms such as ADMM. To resolve these issues, the authors propose the ADMM-PnP framework, which for the first time enables a Plug-and-Play method within ADMM with provable convergence. The framework introduces an AC-DC three-stage denoising mechanism that integrates additive Gaussian noise with self-correction (AC), direction-corrected conditional Langevin dynamics (DC), and score-based denoising, while combining constant and adaptive stepsize strategies. This design effectively mitigates manifold mismatch and simultaneously ensures geometric consistency and algorithmic convergence. Experiments demonstrate that the proposed method consistently achieves superior solution quality over existing baselines across various inverse problems, empirically validating its theoretical convergence guarantees.

This work investigates the convergence properties and implicit optimization objectives of Plug-and-Play (PnP) methods when analytical or synthesis Gaussian denoisers replace the proximal operator in forward–backward (FB) algorithms. Within a dictionary learning framework, we establish—rigorously for the first time—that FB-PnP yields identical solutions regardless of whether the internal denoising sub-iterations are performed once or to convergence. Specifically, synthesis denoisers naturally realize exact proximal mappings, while analysis denoisers, under a warm-restart strategy, become equivalent to primal–dual algorithms. Theoretically, FB-PnP convergence is shown to be independent of the number of inner denoising iterations; a unified analysis is provided via Moreau–Yosida regularization and dual-domain FB expansion. Experiments on compressed sensing and deep dictionary-based image restoration demonstrate both high reconstruction accuracy and strong algorithmic stability.

This work addresses the limited applicability of Bayes-GAMP in complex-valued settings and under arbitrary nonlinear observation models, where the required posterior denoisers typically lack closed-form expressions. By characterizing the message-passing dynamics of GAMP through state evolution, the authors introduce a score-matching framework to train a neural network that replaces analytically intractable denoisers. This approach enables approximate Bayesian-optimal inference using only forward evaluations of the observation mapping, without requiring explicit knowledge of its functional form. Notably, it is the first method to support nearly arbitrary complex-valued nonlinear observation models, eliminating dependence on closed-form denoisers or explicit model specifications. Under ideal training conditions, the proposed algorithm asymptotically approaches the performance of the true Bayes-GAMP, substantially broadening the scope of GAMP-based inference.

This paper addresses model-free denoising of scalar signals corrupted by Gaussian noise. Methodologically, it introduces a family of progressive denoisers grounded in optimal transport theory, which depend solely on higher-order score functions of the observed distribution. By establishing, for the first time, a combinatorial structural connection between higher-order score functions and optimal transport maps—characterized recursively via Bell polynomials—the method achieves hierarchical approximation of the true signal distribution. It requires no prior assumptions on the signal distribution; instead, it estimates higher-order score functions using Gaussian kernel smoothing and score matching, achieving asymptotic optimality under the Wasserstein distance. The limiting transport map (T_infty) strictly satisfies the pushforward condition (T_infty# Q = P). Theoretically, two higher-order score estimation strategies are proven to converge at explicit rates, yielding the first optimal transport-based denoising framework that simultaneously guarantees convergence and distribution-free operation.

Traditional plug-and-play (PnP) methods approximate the maximum a posteriori (MAP) solution using MMSE denoisers, often leading to reconstruction distortions; conversely, direct MAP optimization frequently converges to cartoonish results due to score estimation errors. To address these limitations, this work proposes ProxiMAP, a novel iterative MAP approximation framework that dynamically adjusts the noise schedule to align the residual noise at each iteration with the denoiser’s training distribution. This alignment ensures the denoiser operates within its reliable regime and implicitly enforces early stopping. Based on this principle, we design a plug-and-play ProxiMAP module and a computationally efficient hybrid variant. Experiments demonstrate that our approach significantly enhances reconstruction sharpness and quality across diverse inverse problems—including deblurring, inpainting, super-resolution, and phase retrieval—with the hybrid version achieving comparable or superior performance to full substitution schemes at substantially lower computational cost.

To address the limited sample quality and diversity of score-matching generative models for structured distribution modeling, this paper proposes a novel framework integrating nonlinear forward dynamics with score matching in the VAE latent space. Our key contributions are: (1) replacing conventional linear stochastic differential equations (SDEs) with nonlinear denoising score matching in the VAE latent space; (2) reformulating the cross-entropy regularization term to mitigate gradient variance explosion under small step sizes; and (3) adopting the Euler–Maruyama scheme to approximate Gaussian transitions, balancing accuracy and computational efficiency. Experiments on MNIST variants demonstrate that our method significantly improves generation speed, Fréchet Inception Distance (FID), and diversity metrics—outperforming standard latent-space generative models. This work establishes a new paradigm for efficient, high-fidelity modeling of structured data.