Decomposed Entailment for Factuality Checking and Hallucination Detection

2026-08-06

AI Summary

This work addresses the challenge of factual inconsistency and hallucination in large language model (LLM) outputs, for which existing detection methods often rely on reference texts or white-box access. To overcome these limitations, we propose HallDetect—a lightweight, reference-free hallucination detection framework applicable to black-box models. HallDetect decomposes generated text into atomic claims and employs a contrastive learning–based entailment model to verify each claim against multi-scale fragments of the source text. An asymmetric scoring mechanism is introduced to identify contradictions and assess overall factuality. The framework supports cross-task generalization, offers interpretable error localization, and demonstrates superior performance on three out of four benchmarks compared to state-of-the-art generative and embedding-based baselines under identical hardware conditions and 4-bit quantization, while maintaining consistent results across diverse backbone models.

Abstract

The reliability of Large Language Models (LLMs) is often compromised by factual inconsistencies, including hallucinations---cases where generated content is not supported by the underlying source. We present HallDetect, a lightweight, reference-free, and black-box framework for hallucination detection that we evaluate not only on summarization but across a broader range of source-grounded generation settings. HallDetect builds on decomposition-based factuality evaluation: generated content is decomposed into atomic claims, each verified by a compact encoder-based entailment model through a contrastive formulation over a multi-scale library of source chunks, and aggregated with an asymmetric score in which a single confidently contradicted claim flags the response. Under a controlled protocol in which all methods share the same 4-bit quantized backbones and consumer-grade hardware budget, HallDetect outperforms comparably resourced generative and embedding-based baselines on three of four benchmarks while remaining stable across backbone families, and yields a claim-to-span audit trail that localizes each error.

Problem

• hallucination detection
• factuality checking
• large language models
• source-grounded generation
• factual inconsistency

Innovation

• decomposed entailment
• hallucination detection
• reference-free evaluation
• contrastive verification
• atomic claim decomposition