Two-Step Data Augmentation for Masked Face Detection and Recognition: Turning Fake Masks to Real

2025-12-13

AI Summary

Masked face detection and recognition suffer from scarce real-world labeled data and domain distribution shift. To address these challenges, we propose a two-stage generative data augmentation framework: first, applying geometry-guided controllable warping to the mask region; second, leveraging unpaired GANs for high-fidelity masked-face image synthesis. We introduce a novel non-mask preservation loss and stochastic noise injection to jointly ensure generation diversity, structural consistency, and training stability. This cascaded paradigm uniquely integrates rule-based warping with unpaired image translation—marking the first such integration in this domain. Extensive experiments demonstrate significant improvements over single-stage augmentation methods: +3.2% mAP on WIDER FACE and MAFA for detection, and +4.7% accuracy for recognition. The synthesized images exhibit high visual fidelity and effectively mitigate the scarcity of occluded face samples.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Data scarcity and distribution shift pose major challenges for masked face detection and recognition. We propose a two-step generative data augmentation framework that combines rule-based mask warping with unpaired image-to-image translation using GANs, enabling the generation of realistic masked-face samples beyond purely synthetic transformations. Compared to rule-based warping alone, the proposed approach yields consistent qualitative improvements and complements existing GAN-based masked face generation methods such as IAMGAN. We introduce a non-mask preservation loss and stochastic noise injection to stabilize training and enhance sample diversity. Experimental observations highlight the effectiveness of the proposed components and suggest directions for future improvements in data-centric augmentation for face recognition tasks.

Problem

• Generates realistic masked-face samples for detection
• Addresses data scarcity and distribution shift issues
• Enhances training stability and sample diversity

Innovation

• Two-step generative augmentation combining warping and GAN translation
• Non-mask preservation loss and noise injection for training stability
• Generating realistic masked faces beyond purely synthetic transformations