HyperReCo: Retrieving and Connecting Evidence with Hypergraph Neural Networks for LLM Multi-hop Reasoning

2026-09-26

AI Summary

This study addresses the limitation of existing hypergraph retrieval methods, which overlook query-dependent evidence interactions and rely on implicit evidence connections, thereby constraining the multi-hop reasoning capabilities of large language models. To overcome these challenges, this work proposes a unified framework integrating hypergraph neural networks with retrieval-augmented generation. Specifically, it designs a jointly supervised message-passing mechanism to learn query-aware complementary evidence interactions and introduces a gradient attribution-based hyperpath decoding technique to generate explicit evidence connection paths that facilitate reasoning. Evaluated across six benchmarks, the proposed approach achieves state-of-the-art retrieval performance and substantially improves downstream question answering outcomes, demonstrating the effectiveness of explicit hyperpaths for multi-hop reasoning.

Abstract

Large language models (LLMs) have shown strong capabilities, with retrieval-augmented generation (RAG) supporting complex multi-hop reasoning by retrieving evidence distributed across documents. Graph-based approaches exploit connections among evidence, and hypergraph-based retrieval further preserves higher-order entity associations within documents and connects documents through shared entities. However, existing hypergraph retrievers often rely on predefined structural expansion or diffusion, which may miss query-dependent interactions needed to identify relevant evidence. They also leave connections among retrieved evidence implicit, requiring LLMs to reconstruct these connections before reasoning. Therefore, we propose HyperReCo, a framework for retrieving and connecting evidence with a hypergraph neural network (HyperGNN). We represent each document as a hyperedge over its extracted entities, with shared entities connecting the hyperedges. Through hypergraph message passing with joint supervision over documents and entities, the HyperGNN learns query-dependent interactions to retrieve complementary evidence. We further introduce Gradient-Guided Hyper-Path Decoding (GGHD), which uses gradient attribution to interpret the learned interactions and translate them into explicit hyper-paths that help LLMs combine complementary facts for multi-hop reasoning. Experiments on six benchmarks show that HyperReCo achieves the best retrieval performance among the compared methods on all three multi-hop QA datasets, together with strong downstream QA performance. Case studies and further analyses demonstrate the utility of decoded hyper-paths for connecting retrieved evidence.

Problem

• multi-hop reasoning
• hypergraph retrieval
• retrieval-augmented generation
• evidence connection
• large language models

Innovation

• Hypergraph Neural Networks
• Retrieval-Augmented Generation
• Multi-hop Reasoning
• Gradient-Guided Hyper-Path Decoding
• Large Language Models