Improving Decoupled Posterior Sampling for Inverse Problems using Data Consistency Constraint

2024-12-01

arXiv.org

AI Summary

Existing decoupled posterior sampling methods for diffusion model-based inverse problem solving neglect measurement information during the reverse process, leading to large early-stage errors and unstable optimization. This paper proposes Guided Decoupled Posterior Sampling (GDPS), the first method to explicitly embed data-consistency projection into the reverse process, enabling measurement-driven, smooth optimization transitions. GDPS is further extended to latent diffusion models and integrated with the Tweedie formula to establish a unified estimation framework, significantly enhancing generality and scalability. Evaluated on FFHQ and ImageNet across diverse linear and nonlinear inverse tasks—including super-resolution, denoising, and compressed sensing—GDPS achieves state-of-the-art performance. Notably, it delivers substantial reconstruction accuracy improvements under challenging conditions such as high noise levels and severe undersampling.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Diffusion models have shown strong performances in solving inverse problems through posterior sampling while they suffer from errors during earlier steps. To mitigate this issue, several Decoupled Posterior Sampling methods have been recently proposed. However, the reverse process in these methods ignores measurement information, leading to errors that impede effective optimization in subsequent steps. To solve this problem, we propose Guided Decoupled Posterior Sampling (GDPS) by integrating a data consistency constraint in the reverse process. The constraint performs a smoother transition within the optimization process, facilitating a more effective convergence toward the target distribution. Furthermore, we extend our method to latent diffusion models and Tweedie's formula, demonstrating its scalability. We evaluate GDPS on the FFHQ and ImageNet datasets across various linear and nonlinear tasks under both standard and challenging conditions. Experimental results demonstrate that GDPS achieves state-of-the-art performance, improving accuracy over existing methods.

Problem

• Reducing errors in diffusion models for inverse problems
• Incorporating data consistency in posterior sampling methods
• Improving optimization convergence in reverse diffusion processes

Innovation

• Integrates data consistency constraint in reverse process
• Extends method to latent diffusion models
• Improves accuracy with smoother optimization transition