Likelihood-based Mitigation of Evaluation Bias in Large Language Models

2024-02-25

Annual Meeting of the Association for Computational Linguistics

AI Summary

This work identifies a significant likelihood bias in large language models (LLMs) when used as automatic evaluation metrics: LLMs systematically assign higher scores to surface-level features—such as word order and syntactic structure—favoring high-probability sentences while under-scoring semantically valid yet low-probability ones. To rigorously validate this phenomenon and mitigate the bias, we propose, for the first time, a “bias-to-correct-bias” few-shot in-context learning strategy: prompts are constructed from highly biased examples to calibrate LLM evaluation behavior without fine-tuning. Experiments across data-to-text generation and grammatical error correction tasks, involving multiple state-of-the-art LLMs, confirm the ubiquity of this bias. After calibration, average correlation between LLM scores and human judgments improves by 12.7%; moreover, evaluation robustness and fairness are substantially enhanced.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Philosophy and Ethics of AI: Bias, Fairness & Equity

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large Language Models (LLMs) are widely used to evaluate natural language generation tasks as automated metrics. However, the likelihood, a measure of LLM's plausibility for a sentence, can vary due to superficial differences in sentences, such as word order and sentence structure. It is therefore possible that there might be a likelihood bias if LLMs are used for evaluation: they might overrate sentences with higher likelihoods while underrating those with lower likelihoods. In this paper, we investigate the presence and impact of likelihood bias in LLM-based evaluators. We also propose a method to mitigate the likelihood bias. Our method utilizes highly biased instances as few-shot examples for in-context learning. Our experiments in evaluating the data-to-text and grammatical error correction tasks reveal that several LLMs we test display a likelihood bias. Furthermore, our proposed method successfully mitigates this bias, also improving evaluation performance (in terms of correlation of models with human scores) significantly.

Problem

• LLMs exhibit likelihood bias when evaluating sentences due to superficial differences
• This bias causes overrating high-likelihood sentences and underrating low-likelihood ones
• The paper proposes a mitigation method using biased examples for in-context learning

Innovation

• Uses biased instances as few-shot examples
• Mitigates likelihood bias via in-context learning
• Improves evaluation correlation with human scores