KG-Guard: Graph-Based Hallucination Detection for Knowledge Base Question Answering

2026-05-29

AI Summary

This work addresses the susceptibility of large language models to hallucination in knowledge base question answering (KBQA), where reliance on parametric knowledge often yields unreliable answers. The authors propose the first graph-based hallucination detection method tailored for KBQA, formulating the task as answer node classification within a lightweight graph neural network framework. By introducing a virtual question node and learnable token embeddings to enrich graph representations, the approach integrates knowledge graph semantic embeddings, a graph encoder, and a compact MLP classifier to efficiently verify—via black-box access—whether an answer is grounded in the given subgraph. Evaluated on WebQSP, ComplexWebQuestions, and PUGG, the method achieves F1 scores of 82.0–87.4 with only 1/305 the parameters of baseline models. Its node-level feedback mechanism further boosts downstream KBQA performance, improving F1 by 13.0–14.5 and Exact Match by 16.9–17.6.

Abstract

Large language models (LLMs) are increasingly used for knowledge base question answering (KBQA), where answering requires selecting entities from a question-specific knowledge-graph subgraph. Yet LLMs are known to hallucinate across tasks, and KBQA is no exception: even when we provide a graph as the knowledge source, the model may rely on parametric knowledge instead of graph evidence or perform invalid reasoning over the given relations. Such hallucinated answer nodes can limit the practical deployment of KBQA systems, especially in high-stakes domains such as healthcare. We formulate hallucination detection in KBQA as an answer-node classification problem and propose a lightweight graph-based framework that treats the answering LLM as a black box. \methodname represents each KBQA instance as an augmented graph. It initializes node features with semantic representations of KG entities, marks topic entities and LLM-proposed answer nodes with learned vectors, and connect a virtual question node to the topic entities. A graph encoder then produces verification-oriented node representations, and a small MLP classifies each proposed answer node using its graph representation together with the question embedding. Experiments on WebQSP, ComplexWebQuestions, and PUGG show that our detector achieves the highest F1 on all three benchmarks ($82.0$, $87.4$, and $84.3$), outperforming LLM-as-judge and sampling-based baselines, while having $\sim305\times$ fewer parameters than the reference approaches. Beyond detection, the node-level feedback is actionable: when flagged answers are fed back to the KBQA system for iterative refinement, downstream KBQA F1 improves by $13.0$--$14.5$ points and Exact Match by $16.9$--$17.6$ points.

Problem

• hallucination
• knowledge base question answering
• large language models
• graph-based reasoning
• answer verification

Innovation

• hallucination detection
• knowledge base question answering
• graph-based representation
• black-box LLM
• answer node classification