Efficient Difficulty-Aware Dynamic Routing for Diffusion-Based Real-World Image Super-Resolution

2026-07-17

AI Summary

This work addresses the limitations of existing stable diffusion–based image super-resolution methods, which adopt a uniform processing strategy that disregards variations in image restoration difficulty and suffer from irreversible high-frequency detail loss due to fixed high-ratio VAE downsampling. To overcome these issues, the authors propose a difficulty-aware dynamic routing mechanism that employs a difficulty estimator to predict the restoration cost of each input image and dynamically routes it to Real-ISR networks of varying capacities. Coupled with an adjustable VAE downsampling rate, this approach enables an adaptive trade-off between detail preservation and computational efficiency. As the first to integrate difficulty awareness into diffusion-based super-resolution, the method breaks away from the one-size-fits-all paradigm and achieves state-of-the-art performance across multiple real-world benchmarks, simultaneously enhancing both reconstruction quality and efficiency.

Abstract

Diffusion-based methods have achieved impressive performance in real-world image super-resolution (Real-ISR) by leveraging large pre-trained stable diffusion (SD) models as powerful generative priors. However, these methods still face two key limitations. First, existing SD-based one-step and multi-step Real-ISR approaches adopt a unified processing paradigm for all input samples, ignoring the varying restoration difficulty across images. Second, the aggressive resolution reduction of the VAE in SD models (e.g., 8x downsampling) leads to irreversible loss of fine-scale details, which cannot be recovered by the subsequent diffusion process. To address these limitations, we propose a Difficulty-aware Dynamic Routing (DDR) strategy that overcomes the rigid, one-size-fits-all processing paradigm. Specifically, we first design a difficulty estimator to predict the restoration cost of each input image, enabling automatic assignment to a network of appropriate capacity. Then, we construct a set of Real-ISR networks with varying model capacities by modulating the spatial downsampling ratio of the VAE in the SD backbone, thereby preserving more high-frequency information for challenging cases while maintaining efficiency for simpler inputs. Extensive experiments have demonstrated the superior efficiency and effectiveness of the proposed model compared to recent state-of-the-art methods.

Problem

• real-world image super-resolution
• diffusion models
• restoration difficulty
• VAE downsampling
• detail loss

Innovation

• Difficulty-aware Dynamic Routing
• Diffusion-based Super-Resolution
• VAE Downsampling Modulation
• Adaptive Network Routing
• Real-World Image Super-Resolution