Benchmarking the Explanatory Quality of Open-Weight Vision-Language Models in Face Recognition

2026-09-18

AI Summary

本文针对面部识别中模型解释的质量问题，提出了一种新的评估框架，通过相关性和忠实性两个标准来量化开放权重视觉-语言模型的解释质量。

Abstract

Vision-Language Models (VLMs) have recently been proposed as promising tools for face recognition, as they can produce natural language explanations alongside similarity scores. This capability is considered appealing for face comparisons in forensic contexts, which require decisions to be transparent and auditable. However, existing evaluations of VLMs for that use case focus mostly on recognition accuracy, while the validity of generated explanations remains unquantified. In this work, we introduce a benchmarking framework for VLM-based face recognition that treats explanation quality as a core evaluation axis. We propose two criteria that explanations should satisfy: relevance, i.e., reliance on identity-stable facial features; and faithfulness, i.e., alignment with the visible image content without hallucinated features. We jointly develop a methodology enabling the quantification of relevance and faithfulness of evaluated models, based on constraining model outputs to a structured explanation format that supports automated querying and auditing. Using this framework, we benchmark several families of open-weight VLMs, jointly evaluating face verification accuracy and explanation quality. Our results highlight remaining shortcomings of produced explanations, and emphasize the need for such explanation quality metrics to get a complete picture of model performance. The proposed benchmark and open-source evaluation harness provide a foundation for proper benchmarking and future fine-tuning of explainable face recognition systems.

Problem

• Explanatory Quality
• Vision-Language Models
• Face Recognition
• Relevance
• Faithfulness

Innovation

• explanation quality
• relevance and faithfulness
• benchmarking framework