rank fusion

This study investigates whether retrieval fusion techniques—commonly adopted in real-world retrieval-augmented generation (RAG) systems, such as multi-query retrieval and reciprocal rank fusion—consistently improve end-to-end answer quality under practical deployment constraints. Conducted within an enterprise knowledge-base RAG pipeline, the evaluation is performed under fixed retrieval depth, reranking budget, and latency limits. While retrieval fusion enhances initial recall, it fails to translate into improved Top-k accuracy after subsequent reranking and context truncation; notably, Hit@10 declines from 0.51 to 0.48 and incurs additional latency. These findings challenge the prevailing assumption of the default efficacy of recall-oriented fusion strategies, revealing diminishing returns in production settings where downstream processing and system constraints critically shape overall performance.

This paper addresses the challenge of fusing multi-source heterogeneous evidence in retrieval-augmented generation (RAG). To tackle the incompatibility of disparate scoring scales across IR models and the limited generalization of single-model retrievers, we propose a hierarchical rank fusion framework: (1) constructing dual retrieval channels—one for labeled and one for unlabeled data; (2) unifying retrieval scores via z-score normalization to harmonize heterogeneous ranking outputs; and (3) integrating cross-source results using a multi-information-source separation–aggregation strategy. Our approach effectively mitigates score incomparability and model-specific bias. Empirical evaluation on fact verification demonstrates consistent superiority over state-of-the-art single-model and single-source baselines in retrieval accuracy, while significantly improving out-of-domain generalization. These results validate the efficacy of co-modeling heterogeneous data and jointly optimizing multiple rankers within a unified fusion architecture.

Existing hybrid search systems lack systematic empirical analysis of trade-offs among lexical and semantic retrieval components—i.e., retrieval paradigms, fusion strategies, and re-ranking methods—leading to complex, suboptimal configurations. Method: We introduce the first benchmark framework tailored for advanced hybrid architectures, conducting systematic evaluation across 11 real-world datasets, covering four retrieval paradigms, their combinations, and re-ranking strategies. Contribution/Results: We identify a “weakest-link” effect in hybrid pipelines and propose a data-driven configuration mapping method. Crucially, we find Tensor-based Re-ranking Fusion (TRF) achieves both high efficiency and strong semantic modeling under low-resource conditions, overcoming traditional fusion bottlenecks. Experiments reveal that hybrid performance is severely constrained by imbalanced path quality; optimal configurations are highly dependent on dataset characteristics and resource constraints. TRF significantly improves the effectiveness–cost trade-off, outperforming state-of-the-art baselines across diverse settings.

Fragmented retrieval and RAG tooling, coupled with the absence of a unified evaluation framework, hinders fine-grained experimentation and fair benchmarking. To address this, we introduce RecallKit: a lightweight, modular, end-to-end Python toolkit featuring a novel plug-and-play unified architecture. It seamlessly integrates sparse and dense retrieval, state-of-the-art re-ranking (Cross-Encoder and Bi-Encoder), and RAG generation orchestration. RecallKit provides standardized evaluation interfaces, pre-indexed benchmark datasets, and native compatibility with mainstream components—including Hugging Face models, FAISS/Annoy indexes, and LLM APIs. Released open-source via PyPI, GitHub, and ReadTheDocs, it significantly lowers barriers to developing, reproducing, and evaluating retrieval systems. Empirically, RecallKit has enabled rigorous method comparisons and rapid prototyping across multiple research projects.

This study identifies a performance breakpoint and semantic failure in cross-encoder re-rankers (e.g., ColBERTv2, RankT5) for large-scale document re-ranking: retrieval quality degrades significantly when the candidate set exceeds ~1,000 documents—MRR@10 drops by 12.7% on average, and 38% of top-scoring results exhibit neither lexical overlap nor semantic similarity with the query. Through systematic ablation and scaling experiments, augmented with semantic similarity and lexical matching analyses, we empirically challenge the widely held assumption that re-rankers universally outperform first-stage retrievers. Our key contributions are: (1) establishing the effective scale boundary for cross-encoder re-rankers; (2) revealing their propensity for relevance misjudgment under ultra-large candidate lists; and (3) providing theoretical grounding and practical guidance—along with critical deployment warnings—for integrating re-ranking modules into large-scale retrieval systems.

This work addresses the challenge of effectively fusing heterogeneous scores—such as vector similarity and graph-based relevance measures like personalized PageRank—in graph-augmented retrieval, where distributional mismatches hinder integration. To resolve this, the authors propose a calibration method based on Percentile Rank (PIT) normalization, which maps disparate scores onto a unified, dimensionless scale while preserving magnitude information and enabling stable alignment. Combined with linear and Boltzmann fusion strategies, the approach significantly improves last-hop retrieval performance in multi-hop question answering. On MuSiQue and 2WikiMultiHopQA benchmarks, it achieves LastHop@5 scores of 76.5% and 53.6%, respectively, substantially outperforming existing baselines.

This study addresses the lack of systematic evaluation of modern retrieval methods in Retrieval-Augmented Generation (RAG) systems for heterogeneous financial documents containing both text and tables. It presents the first large-scale benchmark comparing ten retrieval strategies—including sparse, dense, hybrid fusion, cross-encoder reranking, and query expansion—on a financial mixed-document question answering task. Evaluation employs multiple metrics (Recall@k, MRR, nDCG, and numerical match accuracy) alongside bootstrap significance testing. Results reveal that BM25 outperforms state-of-the-art dense retrievers in this domain; a two-stage hybrid retrieval pipeline with neural reranking achieves substantial gains (Recall@5 = 0.816, MRR@3 = 0.605); context-aware retrieval consistently improves performance, whereas query expansion yields limited benefits for numerical questions. The authors release a comprehensive open-source benchmark and provide practical guidance on cost–accuracy trade-offs.

This work addresses the limitations of traditional retrieval-augmented generation (RAG) systems, which rely on static retrieval strategies and fail to dynamically select the optimal retriever for each query. Existing routing approaches further suffer from considering only semantic relevance while neglecting the actual utility of retrieved results in supporting answer correctness. To overcome these issues, the authors propose the R³AG framework, which decouples retriever capability into two learnable dimensions—“retrieval quality” and “generation utility”—and employs contrastive learning with dual supervision signals at both document and answer levels to model the dynamic alignment between queries and retriever capabilities. Experiments across multiple knowledge-intensive tasks demonstrate that R³AG significantly outperforms the strongest single retriever and current static routing methods, validating the effectiveness and superiority of its dynamic routing mechanism.

This work addresses the limitations of single-retrieval approaches in semantic and keyword matching for large language model agents operating in local-first settings. We propose a hybrid retrieval framework that constructs a local document memory system using SQLite, integrating vector similarity search (via sqlite-vec) with full-text keyword matching (FTS5). The system employs an adaptive Reciprocal Rank Fusion (RRF) strategy combined with per-query IDF weighting. Innovatively, we introduce a self-supervised embedding fine-tuning method based on retrieval disagreement, optimizing BGE-small with MultipleNegativesRankingLoss. The framework also incorporates integrity verification, version control, and diagnostic tools. Experiments demonstrate up to a 19.5% improvement in NDCG@10 across five BEIR datasets, with our 33M-parameter model matching or surpassing the 110M-parameter ColBERTv2 and BGE-base on three datasets, while achieving a median retrieval latency of only 20.9ms over 50,000 text chunks.