RELD: Regularization by Latent Diffusion Models for Image Restoration

2025-03-28

AI Summary

This work addresses the longstanding trade-off between perceptual quality and computational efficiency in image restoration tasks—including denoising, deblurring, and super-resolution. We propose embedding a pre-trained latent diffusion model (LDM) as a universal implicit prior into a variational optimization framework. Methodologically, we pioneer the replacement of handcrafted regularizers with the LDM prior and employ Half-Quadratic Splitting for efficient, scalable inference—thereby fully leveraging the LDM’s powerful generative prior while maintaining low computational overhead. Experiments demonstrate state-of-the-art performance across multiple restoration benchmarks, particularly excelling in perceptual metrics such as LPIPS, while preserving both reconstruction fidelity and visual realism. Our approach establishes a lightweight, task-agnostic, and extensible paradigm for generative-prior-based image restoration.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

In recent years, Diffusion Models have become the new state-of-the-art in deep generative modeling, ending the long-time dominance of Generative Adversarial Networks. Inspired by the Regularization by Denoising principle, we introduce an approach that integrates a Latent Diffusion Model, trained for the denoising task, into a variational framework using Half-Quadratic Splitting, exploiting its regularization properties. This approach, under appropriate conditions that can be easily met in various imaging applications, allows for reduced computational cost while achieving high-quality results. The proposed strategy, called Regularization by Latent Denoising (RELD), is then tested on a dataset of natural images, for image denoising, deblurring, and super-resolution tasks. The numerical experiments show that RELD is competitive with other state-of-the-art methods, particularly achieving remarkable results when evaluated using perceptual quality metrics.

Problem

• Integrates Latent Diffusion Model for image restoration
• Reduces computational cost while maintaining quality
• Competes with state-of-the-art in perceptual metrics

Innovation

• Latent Diffusion Model for denoising
• Variational framework with Half-Quadratic Splitting
• Reduced cost with high-quality results