Degradation-Aware Prompt Learning with Cross-Modal Compensation for Adverse Weather Removal

2026-08-07

AI Summary

This work addresses the diverse and complex image degradations caused by adverse weather conditions, which severely impair visual system performance. Existing unified restoration methods often lack explicit spatial and semantic modeling of degradation characteristics. To overcome this limitation, we propose DCMPC-Net, which introduces cross-modal semantic prompting into image restoration for the first time. Our approach leverages a pretrained vision-language model to generate degradation-aware prompts and incorporates a prompt-guided attention alignment mechanism alongside a dual-path feature compensation strategy. This enables context-aware restoration and structural fidelity within a unified backbone architecture. Extensive experiments demonstrate that our method significantly outperforms current state-of-the-art techniques across multiple adverse weather conditions, achieving superior restoration accuracy and visual quality in both task-specific and unified evaluation settings.

Abstract

Adverse weather causes diverse and complex image degradations, severely compromising the reliability of computer vision systems. Existing all-in-one restoration models attempt to address multiple degradation types within a unified framework, but often lack explicit spatial and semantic modeling of degradation characteristics, limiting their adaptability to diverse weather conditions. To address this limitation, we propose a Degradation-Aware Cross-Modal Prompt Compensation Network (DCMPC-Net) that leverages cross-modal degradation cues from a pretrained vision-language model to condition restoration features within a unified backbone. Specifically, our DCMPC-Net mainly consists of the Cross-Modal Prompt Generator (CMPG), Prompt-Guided Attention Alignment Module (PGAAM), and Dual Feature Compensation Module (DFCM). The CMPG integrates textual embeddings with visual features to produce degradation-aware prompts that encode degradation-related semantic and contextual cues. These prompts are injected into the decoder via a PGAAM, which adaptively aligns semantic information with degraded regions to facilitate context-aware restoration. To further enhance structural fidelity, DFCM is introduced that disentangles degradation artifacts from scene structures, thereby improving the reconstruction of fine textures and detailed content. By integrating cross-modal semantic guidance with spatial alignment and structural enhancement, DCMPC-Net achieves robust and perceptually consistent restoration across diverse weather conditions. Extensive experiments show that DCMPC-Net outperforms state-of-the-art methods in both task-specific and unified settings, achieving superior accuracy and visual fidelity.

Problem

• adverse weather removal
• image degradation
• cross-modal compensation
• degradation-aware modeling
• unified restoration

Innovation

• Degradation-Aware Prompt Learning
• Cross-Modal Compensation
• Vision-Language Model
• Attention Alignment
• Feature Disentanglement