Technology Category
Reasoning under Uncertainty: Uncertainty RepresentationsNatural Language Processing: Question AnsweringData Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
SUNAR: Semantic Uncertainty based Neighborhood Aware Retrieval for Complex QA
🤖 AI Summary
In complex question answering, first-stage retrieval suffers from low recall, severely constraining subsequent LLM-based reasoning due to limited context windows—especially under strict depth constraints (k ≤ 10), where performance degrades significantly. To address this, we propose a semantic-uncertainty-driven, neighborhood-aware retrieval framework: (1) an LLM generates answer candidates and quantifies their semantic uncertainty as feedback; (2) a dynamic document neighborhood graph is constructed and iteratively refined to optimize retrieval paths; and (3) retrieval and reasoning signals are tightly coupled in a closed-loop manner. This work is the first to integrate LLM-derived semantic uncertainty into retrieval feedback, enabling lightweight, high-recall retrieval under stringent depth limits. Evaluated on two complex QA benchmarks, our method achieves up to a 31.84% absolute improvement in overall performance, substantially outperforming state-of-the-art approaches.
Technology Category
Application Category
📝 Abstract
Complex question-answering (QA) systems face significant challenges in retrieving and reasoning over information that addresses multi-faceted queries. While large language models (LLMs) have advanced the reasoning capabilities of these systems, the bounded-recall problem persists, where procuring all relevant documents in first-stage retrieval remains a challenge. Missing pertinent documents at this stage leads to performance degradation that cannot be remedied in later stages, especially given the limited context windows of LLMs which necessitate high recall at smaller retrieval depths. In this paper, we introduce SUNAR, a novel approach that leverages LLMs to guide a Neighborhood Aware Retrieval process. SUNAR iteratively explores a neighborhood graph of documents, dynamically promoting or penalizing documents based on uncertainty estimates from interim LLM-generated answer candidates. We validate our approach through extensive experiments on two complex QA datasets. Our results show that SUNAR significantly outperforms existing retrieve-and-reason baselines, achieving up to a 31.84% improvement in performance over existing state-of-the-art methods for complex QA.
Problem
Research questions and friction points this paper is trying to address.
Improving recall in first-stage retrieval for complex QA
Addressing bounded-recall problem with semantic uncertainty
Enhancing retrieval accuracy using neighborhood-aware document exploration
Innovation
Methods, ideas, or system contributions that make the work stand out.
Uses LLMs for Neighborhood Aware Retrieval
Dynamically adjusts document relevance via uncertainty
Iteratively explores document graph for better recall
🔎 Similar Papers
No similar papers found.
