Explore-on-Graph: Hybrid Embedding-LLM Reasoning for Knowledge Graph Question Answering under Incompleteness

2026-09-30

AI Summary

This study addresses the issues of reasoning disruption and evidential hallucination in large language models (LLMs) caused by missing information in knowledge graphs. To this end, we propose XoG, a framework that eliminates reliance on LLM parametric knowledge by recovering missing paths through type-level entity relation statistics and graph embedding retrieval. Furthermore, XoG incorporates an iterative plan-explore-reason mechanism to guide LLMs in semantic selection and multi-hop question answering. Experimental results demonstrate that XoG significantly outperforms existing methods on incomplete knowledge graphs while reducing token consumption by 33%. Moreover, it maintains robust generalization capabilities across diverse LLM backbones.

Abstract

Large language models (LLMs) are increasingly combined with knowledge graphs (KGs) to ground reasoning in structured evidence. However, most LLM-based KGQA methods rely on traversing existing graph edges and become unreliable when reasoning paths are broken by missing facts. Alternatives that ask LLMs to generate missing knowledge risk introducing hallucinated evidence. We introduce XoG (eXplore-on-Graph), a framework for multi-hop question answering over incomplete KGs that recovers missing reasoning paths from learned graph structure rather than LLM parametric knowledge. XoG combines type-level entity-relation statistics to identify candidate relations with KG embeddings to retrieve plausible missing entities, using the LLM as a semantic selector and reasoner. These mechanisms are integrated into an iterative planning-exploration-reasoning process. Experiments on WebQSP, CWQ, and the Wikidata-based BRINK benchmark show that XoG remains competitive on complete KGs and consistently outperforms comparable methods without task-specific KGQA training under KG incompleteness. These gains persist across multiple LLM backbones, indicating that stronger LLMs alone do not resolve missing graph evidence. XoG also reduces LLM token consumption by up to 33% compared with a closely related planning-based approach.

Problem

• Knowledge Graph Question Answering
• Incomplete Knowledge Graphs
• Multi-hop Reasoning
• Hallucination
• Large Language Models

Innovation

• Knowledge Graph Question Answering
• Incomplete Knowledge Graphs
• Hybrid Embedding-LLM
• Multi-hop Reasoning
• Iterative Planning-Exploration-Reasoning