Problem
Research questions and friction points this paper is trying to address.
Face Image Quality Assessment
Error-versus-Discard Characteristic
evaluation reliability
threshold drift
test-set divergence
Beyond Error-vs-Discard Characteristic: Toward Stable and Reliable Evaluation for Face Image Quality Assessment
🤖 AI Summary
Existing face image quality assessment (FIQA) methods based on the error-versus-discard characteristic (EDC) protocol suffer from unreliable evaluation outcomes and limited cross-method comparability due to test set divergence and threshold drift. This work systematically uncovers the fundamental flaws inherent in the EDC protocol and introduces Rank Consistency Evaluation (RCE), a novel paradigm that operates over the full test set without discarding samples. Additionally, it proposes an improved discard-based EDC variant alongside dedicated RCE metrics. Extensive experiments across five datasets, four face recognition models, and fifteen state-of-the-art FIQA methods demonstrate that the proposed approach substantially enhances evaluation stability, reliability, and inter-method comparability.
📝 Abstract
Face Image Quality Assessment (FIQA) aims to estimate the utility of facial images for reliable recognition. The evaluation of FIQA methods is predominantly based on the Error-versus-Discard Characteristic (EDC), which evaluates performance by progressively discarding low-quality samples and measuring recognition error on the retained subset. In this work, we demonstrate that the widely used EDC protocol has fundamental limitations: Test-Set Divergence and Threshold Drift, which together limit the reliability and comparability of FIQA methods. To address this, we propose discard-based EDC variants and a rank-based Rank Consistency Evaluation (RCE) metric that operates on the entire test set without discarding samples, using a fixed decision threshold. Extensive experiments on five datasets, four face recognition models, and 15 state-of-the-art FIQA methods demonstrate both the limitations of EDC and the effectiveness of the proposed approaches in enabling a more reliable and comparable evaluation. Despite evaluated on face images only, the limitations arise from the EDC protocol rather than the biometric modality, suggesting a broader applicability to biometric quality assessment in general.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Face Image Quality Assessment
Error-versus-Discard Characteristic
Rank Consistency Evaluation
Threshold Drift
Test-Set Divergence
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
