Robust Adverse Weather Removal via Spectral-based Spatial Grouping

2025-07-30

AI Summary

Existing global filtering methods struggle to model the diverse, highly localized image degradations induced by adverse weather conditions. Method: We propose a Spectrum-Guided Spatial Grouping Transformer (SGT) that first decomposes images into high-frequency details and low-frequency structures via spectral decomposition (SVD combined with edge detection); then imposes spatial grouping masks to constrain feature interaction ranges; and finally integrates grouped attention with dual channel-spatial attention to jointly model local distortions and global semantics. Contribution/Results: By explicitly embedding spectral priors into the Transformer architecture, SGT achieves robust adaptation to multiple weather degradations—including rain, fog, and snow. Extensive experiments demonstrate that SGT significantly outperforms state-of-the-art methods on multiple All-in-One benchmarks, exhibiting strong generalization and stability. This work establishes a novel paradigm for restoring images corrupted by complex, heterogeneous degradations.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Multimodal Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Adverse weather conditions cause diverse and complex degradation patterns, driving the development of All-in-One (AiO) models. However, recent AiO solutions still struggle to capture diverse degradations, since global filtering methods like direct operations on the frequency domain fail to handle highly variable and localized distortions. To address these issue, we propose Spectral-based Spatial Grouping Transformer (SSGformer), a novel approach that leverages spectral decomposition and group-wise attention for multi-weather image restoration. SSGformer decomposes images into high-frequency edge features using conventional edge detection and low-frequency information via Singular Value Decomposition. We utilize multi-head linear attention to effectively model the relationship between these features. The fused features are integrated with the input to generate a grouping-mask that clusters regions based on the spatial similarity and image texture. To fully leverage this mask, we introduce a group-wise attention mechanism, enabling robust adverse weather removal and ensuring consistent performance across diverse weather conditions. We also propose a Spatial Grouping Transformer Block that uses both channel attention and spatial attention, effectively balancing feature-wise relationships and spatial dependencies. Extensive experiments show the superiority of our approach, validating its effectiveness in handling the varied and intricate adverse weather degradations.

Problem

• Handling diverse adverse weather image degradations
• Addressing localized distortions in frequency domain
• Balancing feature and spatial relationships for restoration

Innovation

• Spectral decomposition for multi-weather restoration
• Group-wise attention for spatial similarity clustering
• Spatial Grouping Transformer balancing feature relationships