Accelerating Diffusion-based Super-Resolution with Dynamic Time-Spatial Sampling

2025-05-17

AI Summary

Diffusion-based super-resolution methods suffer from excessive iterative steps and high computational cost, while existing acceleration strategies lack task-awareness. To address this, we propose a training-free dynamic spatio-temporal sampling framework. We first uncover the spatio-temporal dependency inherent in high-frequency signal recovery during super-resolution and accordingly design two task-aware mechanisms: Temporal Dynamic Sampling (TDS) and Spatial Dynamic Sampling (SDS). These are integrated with adaptive iteration scheduling, content-aware noise modeling, and dynamic step-size allocation. Our method achieves state-of-the-art performance on multiple benchmarks using only ~50% of the original sampling steps, with MUSIQ score improvements ranging from 0.2 to 3.0. It significantly outperforms general-purpose acceleration approaches, demonstrating superior efficiency and fidelity without requiring additional model training.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Diffusion models have gained attention for their success in modeling complex distributions, achieving impressive perceptual quality in SR tasks. However, existing diffusion-based SR methods often suffer from high computational costs, requiring numerous iterative steps for training and inference. Existing acceleration techniques, such as distillation and solver optimization, are generally task-agnostic and do not fully leverage the specific characteristics of low-level tasks like super-resolution (SR). In this study, we analyze the frequency- and spatial-domain properties of diffusion-based SR methods, revealing key insights into the temporal and spatial dependencies of high-frequency signal recovery. Specifically, high-frequency details benefit from concentrated optimization during early and late diffusion iterations, while spatially textured regions demand adaptive denoising strategies. Building on these observations, we propose the Time-Spatial-aware Sampling strategy (TSS) for the acceleration of Diffusion SR without any extra training cost. TSS combines Time Dynamic Sampling (TDS), which allocates more iterations to refining textures, and Spatial Dynamic Sampling (SDS), which dynamically adjusts strategies based on image content. Extensive evaluations across multiple benchmarks demonstrate that TSS achieves state-of-the-art (SOTA) performance with significantly fewer iterations, improving MUSIQ scores by 0.2 - 3.0 and outperforming the current acceleration methods with only half the number of steps.

Problem

• Reducing computational costs in diffusion-based super-resolution methods
• Optimizing temporal and spatial dependencies for high-frequency signal recovery
• Accelerating super-resolution without extra training cost

Innovation

• Time-Spatial-aware Sampling (TSS) strategy
• Time Dynamic Sampling (TDS) for textures
• Spatial Dynamic Sampling (SDS) adapts to content