ClaimPKG: Enhancing Claim Verification via Pseudo-Subgraph Generation with Lightweight Specialized LLM

2025-05-28

AI Summary

Existing approaches for claim verification leveraging knowledge graphs (KGs) and large language models (LLMs) suffer from tight module coupling, poor generalization, and ineffective KG reasoning due to unadapted integration. Method: We propose ClaimPKG, an end-to-end framework featuring a lightweight, task-specific LLM that learns to encode claims as structured “pseudo-subgraphs”; these representations enable efficient KG subgraph retrieval, which is then fused into a general-purpose LLM for joint verification and explanation generation. Contribution/Results: ClaimPKG achieves zero-shot transfer to non-KG datasets (e.g., HoVer, FEVEROUS) without fine-tuning. On FactKG, it outperforms state-of-the-art baselines by 9–12% in accuracy and supports diverse LLM backbones. It is the first method to realize learnable, structured mapping from natural-language claims to KG topology and to enable seamless joint reasoning over structured (KG) and unstructured (LLM) knowledge.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Knowledge Representation and Reasoning: Knowledge Representation Languages

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Integrating knowledge graphs (KGs) to enhance the reasoning capabilities of large language models (LLMs) is an emerging research challenge in claim verification. While KGs provide structured, semantically rich representations well-suited for reasoning, most existing verification methods rely on unstructured text corpora, limiting their ability to effectively leverage KGs. Additionally, despite possessing strong reasoning abilities, modern LLMs struggle with multi-step modular pipelines and reasoning over KGs without adaptation. To address these challenges, we propose ClaimPKG, an end-to-end framework that seamlessly integrates LLM reasoning with structured knowledge from KGs. Specifically, the main idea of ClaimPKG is to employ a lightweight, specialized LLM to represent the input claim as pseudo-subgraphs, guiding a dedicated subgraph retrieval module to identify relevant KG subgraphs. These retrieved subgraphs are then processed by a general-purpose LLM to produce the final verdict and justification. Extensive experiments on the FactKG dataset demonstrate that ClaimPKG achieves state-of-the-art performance, outperforming strong baselines in this research field by 9%-12% accuracy points across multiple categories. Furthermore, ClaimPKG exhibits zero-shot generalizability to unstructured datasets such as HoVer and FEVEROUS, effectively combining structured knowledge from KGs with LLM reasoning across various LLM backbones.

Problem

• Enhancing claim verification using knowledge graphs and LLMs
• Addressing limitations of unstructured text in KG reasoning
• Improving LLM adaptation for multi-step KG reasoning

Innovation

• Lightweight LLM generates pseudo-subgraphs for claims
• Retrieves relevant KG subgraphs for verification
• General-purpose LLM processes subgraphs for final verdict