UniLDiff: Unlocking the Power of Diffusion Priors for All-in-One Image Restoration

2025-07-31

AI Summary

All-in-One image restoration (AiOIR) confronts two key challenges: modeling diverse degradation types and preserving fine details under the high compression ratios and iterative sampling inherent to latent diffusion models (LDMs). To address these, we propose the first unified LDM framework explicitly incorporating degradation priors. Our method introduces a Degradation-Aware Feature Fusion (DAFF) module to dynamically model degradation characteristics, coupled with a Detail-Aware Expert Module (DAEM) that enhances high-frequency reconstruction in the latent space. Without task-specific architectural modifications, the framework adaptively handles both single and mixed degradations. It achieves state-of-the-art performance across denoising, deblurring, super-resolution, and deraining. Experiments demonstrate that integrating structured degradation priors significantly improves the generalizability and detail fidelity of LDMs within a unified architecture, validating the effectiveness of the “diffusion prior + degradation-aware” paradigm.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

All-in-One Image Restoration (AiOIR) has emerged as a promising yet challenging research direction. To address its core challenges, we propose a novel unified image restoration framework based on latent diffusion models (LDMs). Our approach structurally integrates low-quality visual priors into the diffusion process, unlocking the powerful generative capacity of diffusion models for diverse degradations. Specifically, we design a Degradation-Aware Feature Fusion (DAFF) module to enable adaptive handling of diverse degradation types. Furthermore, to mitigate detail loss caused by the high compression and iterative sampling of LDMs, we design a Detail-Aware Expert Module (DAEM) in the decoder to enhance texture and fine-structure recovery. Extensive experiments across multi-task and mixed degradation settings demonstrate that our method consistently achieves state-of-the-art performance, highlighting the practical potential of diffusion priors for unified image restoration. Our code will be released.

Problem

• Unified framework for diverse image restoration tasks
• Mitigating detail loss in latent diffusion models
• Adaptive handling of multiple degradation types

Innovation

• Latent diffusion models for unified restoration
• Degradation-Aware Feature Fusion module
• Detail-Aware Expert Module enhances recovery