face recognition

Human and machine errors in face recognition exhibit unexplored complementarity, hindering the design of reliable human–AI collaborative systems. Method: Through a large-scale, demographically balanced user study, we systematically compare error patterns between human annotators and two state-of-the-art face recognition models. We introduce a novel human–machine verification paradigm grounded in error-feature matching, integrating controlled experimental design, error clustering analysis, and cross-validation. Contribution/Results: We uncover structural disparities in error profiles: machines struggle with cross-race similar faces, whereas humans are more sensitive to illumination and pose variations; furthermore, error biases differ significantly across demographic groups. Our framework improves overall accuracy by 12.3%, substantially reduces high-risk false rejection rates, and provides both theoretical foundations and practical guidelines for trustworthy human–AI fusion in face recognition.

Face recognition (FR) continues to face critical challenges, including limited scalability, difficulty in multimodal fusion, absence of synthetic identity generation, and poor interpretability. This paper systematically reviews fifty years of FR evolution and, for the first time, quantitatively characterizes the impact of data scale and diversity on generalization performance. We identify synthetic identity generation, multimodal fusion, and interpretability as the three core directions for next-generation FR. Methodologically, we integrate CNN and Transformer architectures, employ ArcFace-based contrastive learning, and synergistically train on both real-world and AI-generated facial data, rigorously evaluated under the NIST FRVT benchmark. Experimental results demonstrate state-of-the-art performance: a 0.13% false rejection rate in 1:N million-scale identification on a 12.4M-face gallery, surpassing human accuracy across both high- and low-quality image conditions.

Existing face recognition models exhibit insufficient robustness under Out-of-Distribution (OOD) conditions, particularly degrading significantly when confronted with real-world image degradations and appearance variations. To address this, we introduce OODFace—the first OOD-robustness benchmark specifically designed for face verification—comprising 30 common degradation types (e.g., noise, blur, compression) and appearance variations (e.g., pose, illumination, masks, makeup), and establishing three evaluation splits: LFW-C/V, CFP-FP-C/V, and YTF-C/V. We propose the first dual-dimensional (degradation + appearance) modeling framework for facial OOD challenges, along with a scalable, unified evaluation toolkit. Extensive experiments reveal that 19 open-source models and 3 commercial APIs suffer over 40% accuracy drops under occlusion, illumination shifts, and mask perturbations. We further validate the efficacy of vision-language model–assisted analysis and physical-world experiments. All code, data, and evaluation tools are publicly released.

Existing robustness evaluation methods for face recognition rely heavily on domain expertise, are time-consuming, and suffer from strong system coupling. Method: This paper proposes RobFace—the first system-agnostic, lightweight test suite for face recognition robustness assessment. Its core innovations include: (1) a transferable adversarial face image generation framework enabling cross-model generalization evaluation in black-box settings; (2) a multi-granularity evaluation framework covering perturbation types, intensities, and semantic dimensions; and (3) integration of empirical testing with formal analysis to ensure assessment consistency. Results: Experiments across multiple mainstream face recognition systems demonstrate that RobFace’s evaluation scores correlate strongly with actual attack success rates (Pearson > 0.92), while reducing evaluation time by over 90% compared to state-of-the-art tools. RobFace thus establishes a practical, third-party robustness benchmark—filling a critical gap in the field.

Current deepfake technologies lack systematic structural analysis and quantitative assessment of their impact on facial biometrics. Method: This study establishes the first unified analytical framework for face-oriented deepfakes, integrating generative modeling (GANs, VAEs, diffusion models), explainable detection feature engineering, and robustness evaluation of biometric traits across generation, detection, and recognition stages. Grounded in a decade-long (2014–2024) literature and technological evolution review, the framework systematically categorizes beneficial and harmful applications, identifies seven core technical challenges, and proposes corresponding research pathways. Contribution/Results: It formally defines ethical boundaries for deepfake technologies and pinpoints critical research gaps. The framework has been adopted in AI governance white papers across multiple countries, providing foundational theory and methodological support for deepfake regulation and trustworthy facial recognition systems.

This study addresses the dual challenge in face verification systems: defending against presentation attacks (e.g., photos, videos) while maintaining robustness to legitimate appearance variations such as accessories, lighting, and pose changes. The authors propose a unified framework that, for the first time, integrates classical handcrafted features—including PCA, LBP, HOG, SURF, and Harris corner detectors—within five fusion strategies: Product Matching (PM), Linear Product Matching (LPM), Hierarchical Product Matching (HPM), Sum Matching (SM), and Hybrid Matching (HM). These strategies jointly optimize spoof detection and recognition robustness across preprocessing and classification stages. Experimental results demonstrate that HPM achieves 94.59% accuracy under mixed spoofing attacks, 81.5%–93.2% under lighting and pose variations, and 91.67% anti-spoofing performance; LPM yields the best anti-spoofing rate (93.2%) but exhibits weaker pose robustness. The work further reveals a quantifiable sensitivity–robustness trade-off between spoof detection and appearance invariance.

This study investigates the compatibility of identity embedding spaces generated by diverse deep neural networks, including both specialized and foundation models, for face representation. Treating face embeddings as point clouds, the authors analyze their geometric structures and introduce low-dimensional affine transformations to align embedding spaces across models. The work provides the first systematic evidence of cross-model alignability and convergence in facial representations, substantially improving performance in verification and identification tasks across heterogeneous models. The proposed alignment method demonstrates strong generalization across multiple datasets, with alignment efficacy exhibiting systematic variation across model families. These findings offer novel insights into model interoperability, ensemble design, and the security of biometric templates.

Facial recognition faces privacy infringement and regulatory compliance risks—e.g., under GDPR—arising from real-world data collection. This work systematically evaluates the feasibility of synthetic facial data and, for the first time, proposes and empirically validates seven core privacy-preserving synthetic-data criteria, including identity leakage prevention, intra-class diversity, and inter-class separability. Leveraging multi-million-sample experiments, we conduct comprehensive evaluation across multiple benchmarks (e.g., CASIA-WebFace), assessing recognition accuracy, identity separation, intra-class variation, and fairness. Results show that the top-performing synthetic datasets—VariFace and VIGFace—achieve 95.67% and 94.91% accuracy, respectively, surpassing the real-world CASIA-WebFace benchmark (94.70%). Moreover, they enable controllable bias mitigation and ethically aligned generation. Our study establishes high-fidelity synthetic facial data as a scientifically sound, technically viable, and ethically necessary alternative paradigm.

为解决面部识别系统在不受控视觉条件下的人口统计学可靠性和鲁棒性问题，本文构建了UFPR-PEs基准数据集，采用巴西政治家的公开视频并标注自报种族/肤色类别进行评估。

为解决手动考勤耗时、易错及易伪造问题，通过构建包含16,234张面部样本的Visage Face数据集，并测试多种面部识别模型，以提高教室环境下的面部识别准确性。