ShadowHack: Hacking Shadows via Luminance-Color Divide and Conquer

2024-12-03

arXiv.org

AI Summary

Image shadows cause luminance attenuation, texture degradation, and chromatic distortion, posing a challenge for holistic multi-attribute restoration. To address this, we propose a luminance–chrominance decoupled restoration paradigm: luminance and texture are jointly recovered in the luminance space, while saturation and hue are independently corrected in the chrominance space. We introduce LRNet—featuring a correction-aware attention mechanism—and CRNet—equipped with cross-modal rectangular attention—to dynamically recalibrate corrupted attention maps. Furthermore, we design a luminance–chrominance space decomposition module and a collaborative optimization strategy. Extensive experiments demonstrate that our method achieves significant improvements over state-of-the-art approaches across multiple benchmark datasets, with notable gains in PSNR (+1.27 dB on ISTD) and SSIM (+0.021 on ISTD), alongside more natural visual restoration quality. The source code is publicly available.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Shadows introduce challenges such as reduced brightness, texture deterioration, and color distortion in images, complicating a holistic solution. This study presents extbf{ShadowHack}, a divide-and-conquer strategy that tackles these complexities by decomposing the original task into luminance recovery and color remedy. To brighten shadow regions and repair the corrupted textures in the luminance space, we customize LRNet, a U-shaped network with a rectified attention module, to enhance information interaction and recalibrate contaminated attention maps. With luminance recovered, CRNet then leverages cross-attention mechanisms to revive vibrant colors, producing visually compelling results. Extensive experiments on multiple datasets are conducted to demonstrate the superiority of ShadowHack over existing state-of-the-art solutions both quantitatively and qualitatively, highlighting the effectiveness of our design. Our code will be made publicly available.

Problem

• Recovering brightness in shadow regions of images
• Repairing deteriorated textures in luminance space
• Reviving vibrant colors distorted by shadows

Innovation

• Divide-and-conquer strategy for shadow issues
• LRNet with rectified attention for luminance recovery
• CRNet using cross-attention for color revival