biometric system evaluation

This study addresses the lack of compliant and practical threshold calibration methods for face verification systems in border control under extremely low false match rates (FMRs), hindered by legal and privacy constraints on real-world data acquisition. It presents the first systematic evaluation of synthetic facial data for calibrating identity-to-live face verification thresholds in high-security scenarios. Through score distribution alignment, cross-domain threshold transfer, and adversarial morphing attack testing, the work demonstrates that synthetic data can approximate real-data calibration behavior under controlled conditions. However, under uncontrolled settings, performance degrades significantly due to tail distribution mismatches, introducing notable security vulnerabilities. The findings reveal that calibration efficacy is highly dataset-dependent, highlighting the limited generalizability of synthetic data in real-world deployment contexts.

Existing biometric suitability assessment frameworks—such as the 1998 comparative table—are severely outdated, failing to reflect technological advances and emerging security threats. Method: We propose the first modern, multimodal suitability reassessment framework integrating expert judgment with empirical evidence. It combines structured surveys of 24 domain experts with uncertainty-aware modeling and cross-modal consistency analysis across 55 publicly available biometric datasets. Contribution/Results: Our framework quantifies dynamic shifts in accuracy, security, and usability across major modalities—e.g., improved face recognition performance but declining fingerprint reliability. Expert consensus is high and strongly aligned with empirical findings. Crucially, we identify previously unrecognized methodological gaps and key points of scholarly disagreement. This work establishes a verifiable, updatable assessment paradigm for biometric selection and pinpoints critical research directions, including robustness enhancement and adversarial-resilient design.

Current biometric authentication research largely overlooks the uncertainty inherent in error rate estimation, leading to distorted performance comparisons. To address this, we propose BioQuake—the first framework for quantifying performance uncertainty in multimodal biometric systems—integrating statistical inference with resampling techniques to establish empirical guidelines linking test set size and estimation reliability. We conduct the first systematic reliability assessment across 62 state-of-the-art datasets spanning eight biometric modalities, revealing substantial bias in many reported SOTA results. We release an open-source, interactive BioQuake web tool enabling visualization of error confidence intervals and cross-dataset reliability benchmarking. This work bridges a critical gap in biometric evaluation by introducing principled uncertainty modeling, providing the community with reproducible, standardized reliability metrics for robust performance assessment.

This study investigates the impact of temporal factors on the stability of facial biometric recognition, challenging the conventional assumption that long-term degradation predominantly drives performance decline. Leveraging over 238,000 longitudinal, multimodal, multi-ethnic, multi-gender, and age-diverse facial samples collected over 2.5 years at the BEZ Center—and processed locally in compliance with GDPR—we conduct temporal pairwise comparison analyses using state-of-the-art algorithms. Results reveal that intra-individual recognition score fluctuations across days significantly exceed those attributable to long-term trends; thus, short-term variability, rather than gradual degradation, emerges as the primary determinant of recognition performance. This finding underscores the necessity of controlled, longitudinal individual monitoring for robust biometric assessment. It introduces a novel paradigm for evaluating biometric stability and provides empirical grounding for enhancing system robustness against temporal variation.

This study addresses the limitations of aggregate accuracy metrics in evaluating facial recognition systems within law enforcement contexts, which often obscure performance disparities across demographic groups and fail to capture true fairness and reliability. The authors propose moving beyond a single accuracy measure by introducing a fairness-aware evaluation framework coupled with a model-agnostic auditing strategy. By analyzing false positive and false negative rates at the subpopulation level, this approach uncovers hidden group-level biases that persist even when overall accuracy appears high. Empirical results demonstrate that systems with comparable aggregate accuracy can exhibit substantially different error distributions across demographic groups, underscoring the necessity and effectiveness of the proposed paradigm for enabling more responsible and equitable deployment of facial recognition technologies.

This study addresses the challenge of identity document image quality assessment in remote identity verification, which critically limits the performance of presentation attack detection (PAD). For the first time, it systematically incorporates capture-related quality metrics from the Open Face Image Quality (OFIQ) standard into identity document image evaluation. The proposed approach employs preprocessing steps—including corner detection, perspective normalization, and foreground masking—to produce accurate and unbiased quality scores. Extensive experiments across four diverse identity document datasets demonstrate that the selected OFIQ metrics substantially enhance the performance of three state-of-the-art PAD methods, thereby establishing a clear and quantifiable relationship between image quality and PAD efficacy.

This study addresses the challenge that complex backgrounds in unconstrained scenarios—such as airport border control—degrade face recognition accuracy and impair the detection of presentation attacks. The authors systematically evaluate the impact of multiple face segmentation methods on four representative recognition models and three attack detection techniques through comprehensive experiments on datasets encompassing both controlled and unconstrained imagery. For the first time, they comprehensively demonstrate the dual role of background removal in simultaneously influencing recognition performance and security mechanisms, showing its significant effects on image quality, identification accuracy, and attack detectability. These findings provide empirical grounding and practical guidance for preprocessing strategies in real-world biometric systems, effectively bridging the critical gap between deployment feasibility and system reliability.

This study addresses a critical gap in face recognition security evaluation by introducing intentional electromagnetic interference (IEMI) as a novel physical threat during the image acquisition phase. Leveraging off-the-shelf radio-frequency equipment, the authors systematically assess the robustness of state-of-the-art face recognition algorithms under IEMI attacks conducted during standard face capture procedures. The work presents the first benchmark dataset comprising paired facial images captured with and without electromagnetic interference, which is publicly released to support further research. Experimental results demonstrate that IEMI significantly degrades recognition performance, exposing a previously overlooked vulnerability at the physical-to-digital interface of biometric systems. This approach establishes a new evaluation framework that extends beyond conventional presentation attacks, offering foundational insights for developing more resilient defenses in real-world biometric deployments.

This work addresses the frequent lack of systematic and credible statistical evaluation in ECE/CS research, which often undermines the persuasiveness of empirical claims. To bridge this gap, we propose a structured statistical evaluation workflow tailored for beginners, integrating classical methods—such as t-tests and ANOVA—with modern nonparametric techniques, including bootstrap resampling, Wilcoxon tests, and Cliff’s delta. The framework spans the entire pipeline from formulating research claims to reporting results, supporting factorial designs, multiple comparison corrections, and simulation-based validation. Accompanying the methodology are fully reproducible Python implementations, illustrative examples, and a pre-submission checklist. This approach substantially enhances the reliability and reproducibility of experimental findings while offering both pedagogical utility and practical guidance for researchers.