Need to Know: Contextual-Integrity-Grounded Query Rewriting for Privacy-Conscious LLM Delegation

2026-06-02

AI Summary

This work addresses the challenge that user queries submitted to large language models often intermingle task-essential content with non-essential sensitive information, and traditional type-based personally identifiable information (PII) redaction methods—lacking contextual awareness—frequently result in either privacy leaks or unintended removal of critical details. To tackle this, the study introduces the theory of contextual integrity into LLM privacy-preserving query rewriting for the first time, proposing a task-oriented mechanism to assess the necessity of sensitive information. It further constructs DelegateCI-Bench, the first task-oriented contextual integrity benchmark encompassing synthetic, real-world, and high-sensitivity medical data. Leveraging this benchmark, an end-to-end query rewriter is trained via reinforcement learning, transforming task-relevant and irrelevant signals into optimizable objectives. This approach significantly enhances task utility while preserving privacy, outperforming on-device baselines by an average of 10.1 points.

Abstract

As LLMs become increasingly woven into everyday workflows, user queries sent to cloud hosted LLMs routinely mix task-essential content with task non-essential sensitive disclosures, yet type based PII redaction is context agnostic and may raise two issues: over disclosing untyped sensitive context and over removing answer bearing spans. We recast privacy preserving query rewriting under Contextual Integrity: a span should be forwarded only if it is necessary for the task. We introduce DelegateCI-Bench, the first task based Contextual Integrity benchmark for privacy-conscious delegation, comprising 3,167 samples that combine high quality synthetic data spanning 11 tasks and 20 task types, WildChat based real user queries, and a medical challenge set with dense sensitive information. Building on this benchmark, we propose a CI-guided reinforcement learning framework that converts essential and non-essential sensitive spans into verifiable optimization signals, and train a query rewriter to preserve task critical information while suppressing unnecessary sensitive disclosure. Experiments show that our learned rewriter achieves the best privacy-utility tradeoff, achieving up to +10.1 average utility over on-device baselines.

Problem

• privacy-preserving query rewriting
• contextual integrity
• sensitive information disclosure
• LLM delegation
• PII redaction

Innovation

• Contextual Integrity
• Query Rewriting
• Privacy-Preserving LLM
• Reinforcement Learning
• DelegateCI-Bench