EP-Mem: Elastic Privacy Memory for Social Relationship-Aware LLM Agents

2026-09-28

AI Summary

This study addresses privacy leakage in LLM agents during long-term social interactions by proposing an elastic privacy memory architecture. The framework reconceptualizes privacy as a user-controllable boundary mechanism, managing individuals and events through hierarchical policies. By integrating a sidecar engine with token-level memory, it enables fine-grained disclosure control across generation, storage, and retrieval processes. Additionally, this work introduces EP-Bench, the first long-term multi-party privacy benchmark supporting cross-session correlation. Experimental results demonstrate that the proposed method achieves 94% accuracy in privacy classification, improves disclosure judgment from 22% to 68%, and reduces privacy leakage by 75.6%, all while maintaining robust retrieval performance and generalization capabilities.

Abstract

Large language model (LLM) agents face critical privacy risks when acting as delegates in human-agent-human communication. To prevent such breaches, agents must understand users'social relationships and adhere to context-dependent social information disclosure boundaries. Current studies on agent memory privacy focus on instantaneous interactions, leaving the long-term relational disclosure problem unexplored. In this paper, we propose EP-Mem, an Elastic Privacy Memory architecture that reframes privacy as user-owned boundary control across social roles. EP-Mem introduces (1) token-level memory driven by user-configurable a privacy policy that stratifies persons and events, combining domain-level default circulation rules with fact-level whitelist/blacklist exceptions; and (2) a pluggable sidecar with a privacy engine that aligns disclosure controls with memory across summary, detail, and boundary granularities, enforced throughout generation, storage, and retrieval. We construct EP-Bench, to our knowledge the first long-term multi-party benchmark with cross-session correlated events for policy-conditioned relational disclosure. Experiments show that EP-Mem achieves 94.0% privacy classification accuracy, improves disclosure-permission judgment from 22% to 68%, and reduces privacy leakage by 75.6%, while maintaining retrieval performance and cross-benchmark generalization.

Problem

• LLM agents
• privacy risks
• social relationships
• information disclosure boundaries
• long-term memory

Innovation

• Elastic Privacy Memory
• LLM Agents
• Token-level Memory
• Pluggable Sidecar
• Social Relationship-Aware