Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domains

2025-05-21

AI Summary

Traditional RAG systems rely on similarity-based retrieval and heuristic re-ranking (e.g., top-k), leading to poor interpretability, weak robustness, and vulnerability to adversarial attacks—especially in high-stakes domains like law and finance. To address these limitations, this paper proposes METEORA, a reasoning-driven, three-stage evidence selection framework. It employs a preference-tuned LLM to generate interpretable selection rationales, applies elbow detection for adaptive evidence pruning, and enhances contextual coverage via neighborhood expansion. A dual-LLM coordination mechanism—comprising a rationale generator and a verifier—unifies the reasoning flow, while integrated toxicity filtering and consistency verification eliminate reliance on top-k heuristics. Evaluated across six benchmark datasets, METEORA reduces evidence usage by 50% and improves accuracy by 33.34%. Under adversarial poisoning, its F1 score rises significantly from 0.10 to 0.44.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Traditional Retrieval-Augmented Generation (RAG) pipelines rely on similarity-based retrieval and re-ranking, which depend on heuristics such as top-k, and lack explainability, interpretability, and robustness against adversarial content. To address this gap, we propose a novel method METEORA that replaces re-ranking in RAG with a rationale-driven selection approach. METEORA operates in two stages. First, a general-purpose LLM is preference-tuned to generate rationales conditioned on the input query using direct preference optimization. These rationales guide the evidence chunk selection engine, which selects relevant chunks in three stages: pairing individual rationales with corresponding retrieved chunks for local relevance, global selection with elbow detection for adaptive cutoff, and context expansion via neighboring chunks. This process eliminates the need for top-k heuristics. The rationales are also used for consistency check using a Verifier LLM to detect and filter poisoned or misleading content for safe generation. The framework provides explainable and interpretable evidence flow by using rationales consistently across both selection and verification. Our evaluation across six datasets spanning legal, financial, and academic research domains shows that METEORA improves generation accuracy by 33.34% while using approximately 50% fewer chunks than state-of-the-art re-ranking methods. In adversarial settings, METEORA significantly improves the F1 score from 0.10 to 0.44 over the state-of-the-art perplexity-based defense baseline, demonstrating strong resilience to poisoning attacks. Code available at: https://anonymous.4open.science/r/METEORA-DC46/README.md

Problem

• Replacing heuristic-based re-ranking with rationale-driven selection in RAG
• Improving explainability and robustness against adversarial content
• Enhancing generation accuracy while reducing chunk usage

Innovation

• Replaces re-ranking with rationale-driven selection
• Uses preference-tuned LLM for rationale generation
• Verifies content consistency with Verifier LLM