After a Decade: Bringing Shadow Removal into the Real World with Agentic Training Data

2026-09-29

AI Summary

This study addresses the limited generalization of existing shadow removal models in real-world scenarios due to the scarcity of diverse paired training data. To overcome this, we propose an offline agent-based closed-loop workflow that pioneers the transformation of untargeted large-scale detection data into high-quality paired supervision signals. Through physics-driven generation, deterministic correction, and multi-stage candidate filtering, we construct AgenticShadow, a comprehensive dataset encompassing general scenes, faces, and remote sensing imagery. This work breaks the decade-long data bottleneck in the field. Experimental results demonstrate that our approach reduces color distribution discrepancies by 50.5% and decreases cross-domain LAB RMSE errors by 19.7%–37.5%, significantly enhancing the real-world generalization performance of shadow removal models.

Abstract

Shadow removal looks nearly solved on established benchmarks, yet remains brittle in the real world. Models have advanced; the paired training data they rely on have barely changed in nearly a decade. The reason is simple: obtaining a shadow-free target requires removing the occluder while keeping the scene, camera, and illumination otherwise unchanged, making diverse paired data difficult to capture. Meanwhile, large shadow detection datasets already contain diverse real-world images and masks, but no shadow-free targets. To turn this abundant but incomplete data into paired supervision, we propose an offline agentic workflow combining physics-motivated generation, failure detection, feedback-driven retry, candidate selection, and deterministic correction. Using this workflow, we construct AgenticShadow, a dataset of 17,138 image-mask-target triplets spanning general scenes, faces, and remote sensing. Our construction workflow reduces Color Distribution Difference by 50.5% over previous shadow removal work, while training existing shadow removal models on AgenticShadow reduces cross-domain LAB RMSE by 19.7-37.5%.

Problem

• shadow removal
• paired training data
• real-world robustness
• shadow detection dataset

Innovation

• Shadow Removal
• Agentic Workflow
• Training Data Generation
• AgenticShadow
• Cross-domain Generalization