FreeShadow: Training-Free Shadow Removal via Illumination Transfer and Selective Content Preservation in Diffusion Models

2026-07-29

AI Summary

This work addresses the limitations of existing shadow removal methods, which often suffer from poor generalization or artifacts due to insufficient training data diversity or time-consuming test-time optimization. The authors propose a training-free, optimization-free zero-shot approach leveraging a pre-trained diffusion model. By integrating an Illumination Transfer Attention (ITA) mechanism and a Local Texture-Preserving Relighting (LTPR) strategy, the method simultaneously achieves accurate illumination recovery and content fidelity without fine-tuning. It exploits the diffusion model’s self-attention maps and latent high-frequency features to effectively preserve local textures while transferring plausible lighting. The approach generates realistic, shadow-free images across diverse scenarios and significantly outperforms both current zero-shot and supervised methods.

Abstract

Existing supervised and unsupervised shadow removal methods often suffer from limited generalization due to the insufficient diversity of available training datasets, while zero-shot methods tend to produce artifacts and require time-consuming test-time optimization. To address these issues, we propose FreeShadow, a training-free shadow removal method built upon pretrained diffusion models, which exploits diffusion priors for shadow removal without any training or optimization. For illumination recovery, we propose an illumination transfer attention (ITA), which re-weights the self-attention maps in diffusion model to transfer illumination cues from non-shadow to shadow regions. For content preservation, we analyze the effects of illumination variations on self-attention maps and latent high-frequency features in diffusion model, and selectively preserve illumination-invariant components to maintain content fidelity while suppressing residual shadows. We further propose local texture-preserving relighting (LTPR) to mitigate local texture misalignment caused by VAE compression. Extensive experiments demonstrate that our method achieves strong generalization and produces realistic shadow-free images.

Problem

• shadow removal
• generalization
• artifacts
• test-time optimization
• illumination recovery

Innovation

• training-free
• diffusion models
• illumination transfer
• content preservation
• shadow removal