Verify-in-the-Graph: Entity Disambiguation Enhancement for Complex Claim Verification with Interactive Graph Representation

2025-05-29

North American Chapter of the Association for Computational Linguistics

AI Summary

To address the degradation in accuracy and interpretability of complex claim verification caused by entity ambiguity, this paper proposes a three-stage verification framework integrating graph representation learning with large language model (LLM) agents. The core innovation is a novel graph-structure-driven iterative entity disambiguation mechanism: claims are first parsed into structured triple-based graphs; then, knowledge graph querying, graph neural network representation learning, and multi-step interactive reasoning via a Meta-Llama-3-70B agent jointly resolve ambiguous or implicit entities in a dynamic, context-aware manner. Evaluated on the HoVer and FEVEROUS benchmarks, the method achieves state-of-the-art performance, improving verification F1 score by +3.2% while significantly enhancing attribution interpretability. It is the first approach to enable end-to-end controllable, traceable entity disambiguation and verifiable reasoning for complex claims containing ambiguous entities.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Knowledge Representation and Reasoning: Argumentation
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Claim verification is a long-standing and challenging task that demands not only high accuracy but also explainability of the verification process. This task becomes an emerging research issue in the era of large language models (LLMs) since real-world claims are often complex, featuring intricate semantic structures or obfuscated entities. Traditional approaches typically address this by decomposing claims into sub-claims and querying a knowledge base to resolve hidden or ambiguous entities. However, the absence of effective disambiguation strategies for these entities can compromise the entire verification process. To address these challenges, we propose Verify-in-the-Graph (VeGraph), a novel framework leveraging the reasoning and comprehension abilities of LLM agents. VeGraph operates in three phases: (1) Graph Representation - an input claim is decomposed into structured triplets, forming a graph-based representation that integrates both structured and unstructured information; (2) Entity Disambiguation -VeGraph iteratively interacts with the knowledge base to resolve ambiguous entities within the graph for deeper sub-claim verification; and (3) Verification - remaining triplets are verified to complete the fact-checking process. Experiments using Meta-Llama-3-70B (instruct version) show that VeGraph achieves competitive performance compared to baselines on two benchmarks HoVer and FEVEROUS, effectively addressing claim verification challenges. Our source code and data are available for further exploitation.

Problem

• Enhancing entity disambiguation for complex claim verification
• Improving explainability in claim verification using graph representation
• Leveraging LLMs to resolve ambiguous entities in knowledge bases

Innovation

• Leverages LLM agents for reasoning and comprehension
• Uses graph representation for structured claim decomposition
• Iteratively disambiguates entities with knowledge base interaction