ScalableRAG: High-Quality RAG at Zero Ingestion Cost

2026-07-27

AI Summary

This work addresses the trade-off between performance and scalability in traditional retrieval-augmented generation (RAG) approaches, which often rely on costly knowledge ingestion processes such as constructing knowledge graphs or SQL tables. The authors propose Zero-Ingestion ScalableRAG, a novel framework that enables real-time aggregative reasoning over readable and writable document and value-set workspaces without requiring vector databases or preprocessing. They further introduce Limited-Ingestion ScalableRAG, which integrates a lightweight vector store with automated schema discovery to enhance accuracy at scale. This approach achieves high-quality RAG inference with zero ingestion cost and ensures scalability through a constant-level LLM invocation mechanism. Evaluated on six benchmark datasets, the method significantly outperforms all baselines—including knowledge graph–based approaches—on three datasets and matches near-optimal performance on the rest, surpassing the second-best baseline by an average accuracy margin of 7.36%.

Abstract

Recent advances in RAG aim to optimize for performance by paying high ingestion costs for knowledge ingestion: building knowledge graphs or extracting SQL tables. In this work we show that the operations that such knowledge bases allow can be replicated with zero ingestion costs (not even a vector database); in fact our solution, Zero-Ingestion ScalableRAG, handily out-performs all baselines (including knowledge graph approaches) in three out of the six corpora considered here, and only marginally missing maximum performance on the other three, with average accuracy across all six datasets 7.36% above the next most competitive baseline. It achieves this by keeping a workspace of document sets and values sets that it can write into and read from, allowing for on-the-fly aggregative reasoning in all situations where grouping is required on a primary key that is in one to one correspondence with a subset of the total document set. Capping the number of LLM calls by a constant independent of the corpus size, we also introduce Limited-Ingestion ScalableRAG, which does use a minimal vector database as well as an automated pattern discovery from a sample of documents, to further improve accuracy at scale. Our code is available at https://github.com/cohesity/ScalableRAG .

Problem

• Retrieval-Augmented Generation
• zero ingestion cost
• knowledge graph
• vector database
• scalable RAG

Innovation

• Zero-Ingestion RAG
• Scalable Retrieval-Augmented Generation
• On-the-fly Aggregative Reasoning
• LLM Call Efficiency
• Minimal Vector Database