Technology Category
Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)Knowledge Representation and Reasoning: Computational Complexity of ReasoningMachine Learning: Evaluation and Analysis
Verify with Caution: The Pitfalls of Relying on Imperfect Factuality Metrics
🤖 AI Summary
Existing factual consistency evaluation metrics exhibit unstable cross-dataset performance and frequently misestimate model capability—particularly under content rewriting or when source information spans long distances. Method: We systematically benchmark five mainstream factuality metrics across 11 summarization, RAG, and question-answering benchmarks, employing multi-metric横向 comparison, cross-task evaluation, and bias attribution experiments—including rewriting sensitivity and source span analysis. Contribution/Results: Our empirical study uncovers four fundamental flaws: systemic bias, poor domain transferability, literal translation preference, and neglect of distant contextual information. Metrics achieve only 0.32 average Spearman correlation with human judgments; 43% of datasets yield incorrect system rankings; and for highly rewritten outputs, misclassification rates exceed 68%. These findings challenge prevailing automatic evaluation practices and motivate a paradigm shift toward “human verification before deployment.”
Technology Category
Application Category
📝 Abstract
Improvements in large language models have led to increasing optimism that they can serve as reliable evaluators of natural language generation outputs. In this paper, we challenge this optimism by thoroughly re-evaluating five state-of-the-art factuality metrics on a collection of 11 datasets for summarization, retrieval-augmented generation, and question answering. We find that these evaluators are inconsistent with each other and often misestimate system-level performance, both of which can lead to a variety of pitfalls. We further show that these metrics exhibit biases against highly paraphrased outputs and outputs that draw upon faraway parts of the source documents. We urge users of these factuality metrics to proceed with caution and manually validate the reliability of these metrics in their domain of interest before proceeding.
Problem
Research questions and friction points this paper is trying to address.
Factuality Evaluation
Language Models
Consistency Issues
Innovation
Methods, ideas, or system contributions that make the work stand out.
Fact Scoring Limitations
Natural Language Generation
Human Verification Necessity
🔎 Similar Papers
No similar papers found.
