Diversify-verify-adapt: Efficient and Robust Retrieval-Augmented Ambiguous Question Answering

2024-09-04

arXiv.org

AI Summary

To address incomplete retrieval, low-quality paragraph selection, and inefficiency of iterative methods in Retrieval-Augmented Generation (RAG) for ambiguous queries, this paper proposes DIVA—a three-stage closed-loop framework comprising diversified retrieval, confidence-driven quality verification, and adaptive generation. DIVA employs semantic diversity sampling to ensure comprehensive coverage of query ambiguities, integrates confidence-based validation to filter low-quality passages, and dynamically routes to specialized adapters (e.g., re-ranking, rewriting, or direct generation) tailored to retrieval outcomes. This design is the first to simultaneously achieve robustness and substantial efficiency gains over conventional iterative RAG. Evaluated on multiple fuzzy question-answering benchmarks, DIVA outperforms state-of-the-art methods by +12.3% in accuracy and reduces inference latency by 58%, markedly improving ambiguity coverage and response stability.

Technology Category

Data Mining & Knowledge Management: Conversational Systems for Recommendation & RetrievalNatural Language Processing: Question AnsweringSearch and Optimization: Distributed Search

Application Category

Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAGUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and rankingSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The retrieval augmented generation (RAG) framework addresses an ambiguity in user queries in QA systems by retrieving passages that cover all plausible interpretations and generating comprehensive responses based on the passages. However, our preliminary studies reveal that a single retrieval process often suffers from low quality results, as the retrieved passages frequently fail to capture all plausible interpretations. Although the iterative RAG approach has been proposed to address this problem, it comes at the cost of significantly reduced efficiency. To address these issues, we propose the diversify-verify-adapt (DIVA) framework. DIVA first diversifies the retrieved passages to encompass diverse interpretations. Subsequently, DIVA verifies the quality of the passages and adapts the most suitable approach tailored to their quality. This approach improves the QA systems accuracy and robustness by handling low quality retrieval issue in ambiguous questions, while enhancing efficiency.

Problem

• Enhance ambiguous question answering accuracy
• Improve retrieval-augmented generation efficiency
• Ensure robustness in diverse passage retrieval

Innovation

• Diversifies retrieved interpretations
• Verifies passage quality
• Adapts approach based quality