CheckEmbed: Effective Verification of LLM Solutions to Open-Ended Tasks

2024-06-04

arXiv.org

AI Summary

To address the challenge of efficiently verifying answer credibility for large language models (LLMs) on open-ended tasks (e.g., summarization, knowledge extraction), this paper proposes a lightweight, answer-level embedding-based automatic verification paradigm. Methodologically, it maps each full answer to a single high-quality text embedding—enabling fine-grained similarity comparison against reference ground truths—and introduces embedding heatmaps and interpretable summary metrics to form an end-to-end trustworthiness assessment framework. The approach leverages GPT Text Embedding Large, combined with similarity computation and threshold-driven decision-making. Evaluated on document analysis tasks, it significantly outperforms BERTScore and SelfCheckGPT in accuracy, inference cost, and runtime speed, achieving concurrent gains in both efficiency and reliability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large Language Models (LLMs) are revolutionizing various domains, yet verifying their answers remains a significant challenge, especially for intricate open-ended tasks such as consolidation, summarization, and extraction of knowledge. In this work, we propose CheckEmbed: an accurate, scalable, and simple LLM verification approach. CheckEmbed is driven by a straightforward yet powerful idea: in order to compare LLM solutions to one another or to the ground-truth, compare their corresponding answer-level embeddings obtained with a model such as GPT Text Embedding Large. This reduces a complex textual answer to a single embedding, facilitating straightforward, fast, and meaningful verification. We develop a comprehensive verification pipeline implementing the CheckEmbed methodology. The CheckEmbed pipeline also comes with metrics for assessing the truthfulness of the LLM answers, such as embedding heatmaps and their summaries. We show how to use these metrics for deploying practical engines that decide whether an LLM answer is satisfactory or not. We apply the pipeline to real-world document analysis tasks, including term extraction and document summarization, showcasing significant improvements in accuracy, cost-effectiveness, and runtime performance compared to existing token-, sentence-, and fact-level schemes such as BERTScore or SelfCheckGPT.

Problem

• Verifying LLM outputs for complex open-ended tasks
• Overcoming accuracy and scalability limitations in verification
• Generalizing verification framework beyond text to other modalities

Innovation

• Uses modern embedding LLM models
• Performs whole-answer level comparisons
• Generalizes to multiple modalities