When Knowledge Is Not Free: Cost-Aware Evidence Selection in Retrieval-Augmented Generation

2026-06-01

AI Summary

This work addresses a critical limitation of traditional retrieval-augmented generation (RAG) systems, which ignore the access costs of external knowledge—such as paywalls or licensing constraints—rendering them impractical in real-world settings. The paper formally introduces the problem of cost-aware RAG, incorporating hierarchical access costs and budget constraints to optimize evidence selection under limited resources while maximizing answer quality. To facilitate research in this direction, the authors construct a cost-annotated benchmark by extending MS MARCO v2.1 and propose a large language model–driven agent that dynamically and adaptively selects evidence based on cost-performance trade-offs. Experimental results demonstrate that this agent consistently achieves a more robust balance between cost and effectiveness compared to static retrieval strategies, yielding significant performance gains across diverse tasks and underlying models.

Abstract

Retrieval-Augmented Generation (RAG) typically assumes that external knowledge is free, but many high-quality sources are paywalled, licensed, restricted, or otherwise costly to access. We introduce cost-aware RAG, a setting where retrieved evidence is assigned access-cost tiers and systems must answer under an explicit evidence-access budget. We instantiate this setting by augmenting MS MARCO v2.1 with access-friction tiers and evaluate budgeted evidence selection across general-domain and domain-specific QA benchmarks. Our results show that static selection is brittle: no fixed selector uniformly dominates, and larger budgets do not reliably improve answer quality, even when costly evidence is domain-matched. We then study agentic cost-aware RAG, where an LLM decides when to retrieve, which tier to access, and when to stop. Agents show strong promise as adaptive evidence-acquisition controllers, but their behavior remains highly model- and task-dependent. These findings suggest that cost-aware evidence acquisition is a central challenge for the next generation of RAG systems. All code and data are available at https://github.com/Mignonmy/Cost-Aware.

Problem

• Retrieval-Augmented Generation
• cost-aware
• evidence selection
• access budget
• knowledge access cost

Innovation

• cost-aware RAG
• evidence selection
• access budget
• retrieval-augmented generation
• agentic retrieval