Arbitrary-steps Image Super-resolution via Diffusion Inversion

2024-12-12

arXiv.org

AI Summary

Diffusion models for image super-resolution (SR) often suffer from a trade-off between sampling efficiency and reconstruction quality. To address this, this paper proposes an efficient SR method leveraging the inverse process of a pre-trained diffusion model. Our approach introduces three key contributions: (1) a novel partial noise prediction mechanism, wherein a deep noise estimator constructs an intermediate latent state serving as a flexible initialization for sampling; (2) inference compatibility with arbitrary step counts (1–5 steps), achieving competitive or superior PSNR/SSIM performance even with a single denoising step; and (3) joint optimization of forward diffusion modeling and reverse sampling, significantly boosting fidelity metrics. Extensive experiments demonstrate state-of-the-art performance across multiple standard benchmarks. The source code and pre-trained models are publicly available.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This study presents a new image super-resolution (SR) technique based on diffusion inversion, aiming at harnessing the rich image priors encapsulated in large pre-trained diffusion models to improve SR performance. We design a Partial noise Prediction strategy to construct an intermediate state of the diffusion model, which serves as the starting sampling point. Central to our approach is a deep noise predictor to estimate the optimal noise maps for the forward diffusion process. Once trained, this noise predictor can be used to initialize the sampling process partially along the diffusion trajectory, generating the desirable high-resolution result. Compared to existing approaches, our method offers a flexible and efficient sampling mechanism that supports an arbitrary number of sampling steps, ranging from one to five. Even with a single sampling step, our method demonstrates superior or comparable performance to recent state-of-the-art approaches. The code and model are publicly available at https://github.com/zsyOAOA/InvSR.

Problem

• Develops image super-resolution via diffusion inversion
• Uses Partial noise Prediction for efficient sampling
• Supports arbitrary sampling steps for flexibility

Innovation

• Uses diffusion inversion for image super-resolution.
• Implements Partial noise Prediction strategy.
• Supports arbitrary sampling steps efficiently.