Global Modeling Matters: A Fast, Lightweight and Effective Baseline for Efficient Image Restoration

2025-07-18

AI Summary

Adverse weather conditions severely degrade natural image quality, hindering downstream vision tasks. To address the high computational cost and poor real-time deployability of existing Transformer-based restoration methods, this paper proposes a lightweight and efficient Pyramid Wavelet–Fourier Network (PW-FNet). PW-FNet replaces costly self-attention with a synergistic design combining wavelet-based multi-scale decomposition and Fourier-domain global modeling: a pyramid wavelet architecture enables multi-input-multi-output multi-scale feature extraction, while iterative multi-band Fourier transforms provide global receptive fields with drastically reduced complexity. Extensive experiments on deraining, dehazing, and super-resolution demonstrate that PW-FNet achieves state-of-the-art performance with 38% fewer parameters, 52% lower FLOPs, and 2.1× faster inference speed—significantly enhancing practical deployment feasibility.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Hardware-aware ML
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Natural image quality is often degraded by adverse weather conditions, significantly impairing the performance of downstream tasks. Image restoration has emerged as a core solution to this challenge and has been widely discussed in the literature. Although recent transformer-based approaches have made remarkable progress in image restoration, their increasing system complexity poses significant challenges for real-time processing, particularly in real-world deployment scenarios. To this end, most existing methods attempt to simplify the self-attention mechanism, such as by channel self-attention or state space model. However, these methods primarily focus on network architecture while neglecting the inherent characteristics of image restoration itself. In this context, we explore a pyramid Wavelet-Fourier iterative pipeline to demonstrate the potential of Wavelet-Fourier processing for image restoration. Inspired by the above findings, we propose a novel and efficient restoration baseline, named Pyramid Wavelet-Fourier Network (PW-FNet). Specifically, PW-FNet features two key design principles: 1) at the inter-block level, integrates a pyramid wavelet-based multi-input multi-output structure to achieve multi-scale and multi-frequency bands decomposition; and 2) at the intra-block level, incorporates Fourier transforms as an efficient alternative to self-attention mechanisms, effectively reducing computational complexity while preserving global modeling capability. Extensive experiments on tasks such as image deraining, raindrop removal, image super-resolution, motion deblurring, image dehazing, image desnowing and underwater/low-light enhancement demonstrate that PW-FNet not only surpasses state-of-the-art methods in restoration quality but also achieves superior efficiency, with significantly reduced parameter size, computational cost and inference time.

Problem

• Restores images degraded by adverse weather conditions
• Reduces computational complexity in image restoration
• Improves efficiency without sacrificing restoration quality

Innovation

• Pyramid wavelet-based multi-scale decomposition
• Fourier transforms replace self-attention mechanisms
• Lightweight network with global modeling capability