ScaleResfusion: Residual Rectified Flow based on Residual Vector Field

2026-07-28

AI Summary

This work addresses key challenges in real-world image restoration with diffusion models—namely slow inference, insufficient fidelity, and ineffective utilization of pretrained generative priors. The authors propose a scalable restoration framework built upon a pretrained text-to-image Rectified Flow model, introducing a residual vector field to construct a residual Rectified Flow that enables efficient transport starting from degraded images rather than pure noise. This approach preserves consistency with the original pretrained model while allowing parameter-efficient fine-tuning. A knowledge distillation strategy is further integrated to reduce sampling costs. Extensive experiments demonstrate state-of-the-art performance across multiple real image restoration tasks, significantly accelerating inference and validating the practicality and scalability of large-scale pretrained diffusion models for restoration applications.

Abstract

Real-world Image Restoration (Real-IR) aims to recover high-quality (HQ) images from complex and unknown degradations. Although recent diffusion-based methods have substantially improved perceptual quality, their current designs leave two key challenges unresolved. Methods that start from Gaussian noise are slow and often less faithful to the degraded input. Residual-based methods usually train from scratch, which makes it hard to exploit modern pre-trained generative priors. In this paper, we present ScaleResfusion, a scalable diffusion framework for real-world image restoration built on pre-trained text-to-image rectified-flow models. The core of our method is Residual Rectified Flow, which introduces the residual term R into Standard Rectified Flow. Instead of starting from pure noise, it uses a residual transport path that starts from noisy low-quality (LQ) images and admits an exact acceleration point. By learning the residual vector field, Residual Rectified Flow keeps the output distribution and linear diffusion process consistent with the pre-trained rectified-flow models. This makes parameter-efficient fine-tuning possible at scale. We further introduce a knowledge-distillation pipeline to reduce sampling cost while maintaining restoration quality. Extensive experiments on multiple real-world restoration tasks show that ScaleResfusion achieves state-of-the-art performance with much higher efficiency. These results suggest a practical and scalable way to adapt large pre-trained diffusion models to real-world image restoration. Our code and models are available at https://github.com/YukinoshitaLove/ScaleResfusion.

Problem

• Real-world Image Restoration
• Diffusion Models
• Residual Learning
• Pre-trained Generative Priors
• Image Degradation

Innovation

• Residual Rectified Flow
• Real-world Image Restoration
• Pre-trained Diffusion Models
• Parameter-efficient Fine-tuning
• Knowledge Distillation