TDiR: Transformer based Diffusion for Image Restoration Tasks

2025-06-25

AI Summary

In challenging environments, images often suffer from multiple degradations—including noise, color cast, blur, and light scattering—posing significant challenges for restoration. To address this, we propose the first Transformer-based diffusion model (TDM) for general-purpose image restoration. TDM integrates visual Transformers into the diffusion process, leveraging self-attention to capture long-range dependencies and hierarchical degradation structures, enabling fine-grained, progressive denoising. Unlike existing task-specific enhancement methods, TDM unifies diverse restoration tasks—including underwater enhancement, image denoising, and deraining—within a single framework. Extensive experiments on multiple public benchmarks demonstrate that TDM achieves state-of-the-art performance in quantitative metrics (PSNR, SSIM, and LPIPS) and significantly improves downstream vision tasks such as object detection and image classification.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Images captured in challenging environments often experience various forms of degradation, including noise, color cast, blur, and light scattering. These effects significantly reduce image quality, hindering their applicability in downstream tasks such as object detection, mapping, and classification. Our transformer-based diffusion model was developed to address image restoration tasks, aiming to improve the quality of degraded images. This model was evaluated against existing deep learning methodologies across multiple quality metrics for underwater image enhancement, denoising, and deraining on publicly available datasets. Our findings demonstrate that the diffusion model, combined with transformers, surpasses current methods in performance. The results of our model highlight the efficacy of diffusion models and transformers in improving the quality of degraded images, consequently expanding their utility in downstream tasks that require high-fidelity visual data.

Problem

• Restores degraded images with noise and blur
• Enhances underwater, denoised, and derained image quality
• Improves downstream task performance via diffusion-transformers

Innovation

• Transformer-based diffusion model for image restoration
• Combines diffusion models with transformers effectively
• Outperforms existing methods in multiple quality metrics