Only Pay What You Must Spend: On-Demand Privacy Budget Payment for Differentially Private RAG

2026-09-23

AI Summary

为解决RAG在敏感数据上部署时的隐私预算快速耗尽问题，提出SparsePay-RAG方法，通过仅对必要私有增量支付隐私预算来优化隐私预算使用。

Abstract

Deploying large language models (LLMs) on sensitive data via Retrieval-Augmented Generation (RAG) introduces severe privacy risks. Recent studies apply Differential Privacy (DP) to LLMs with RAG for formal privacy guarantees. However, existing DP-RAG frameworks rapidly exhaust the privacy budget. Although recent efforts attempt to save the budget by narrowing the retrieval scope or sparsifying private generation, these methods themselves cumulatively consume the budget, whereas they could actually rely merely on public information or at a negligible one-time privacy cost. This mismatch fails to align budget expenditure with the model's actual reliance on private data, causing substantial waste on operations that require no private access. To address this, we propose SparsePay-RAG, adopting "only pay what you must spend" as its core principle. Using public information as a zero-privacy prior, it charges the privacy budget only for the private increment. Specifically, SparsePay-RAG narrows the retrieval scope via public topic-guided clustering, adaptively controls private access frequency without privacy cost through isotonic cross-layer trajectory fitting, and compresses per-access budget via DP contrastive decoding. Under strong privacy constraints, experiments show SparsePay-RAG achieves superior privacy-utility trade-offs over baselines.

Problem

• Differential Privacy
• Retrieval-Augmented Generation
• Privacy Budget

Innovation

• Differential Privacy
• Retrieval-Augmented Generation
• Privacy Budget
• Public Information Prior
• Adaptive Access Control