Privacy in Personalized AI Is a System Property, Not Just a Model Property

2026-09-29

AI Summary

This project addresses the limitation of single-model analyses in capturing system-level privacy risks within personalized AI by departing from traditional component-level paradigms to establish privacy as an emergent system property. Through systematic architectural analysis and privacy risk modeling, it identifies four distinct risk channels and constructs a multidimensional evaluation framework encompassing interaction trajectories, information flows, indirect leakage, and utility-privacy trade-offs. Ultimately, this work delivers actionable, systematized privacy auditing standards and an assessment methodology that effectively bridges critical gaps in existing audit approaches at the system level.

Abstract

In personalized AI applications, such as conversational assistants and recommender systems, users interact not with models in isolation but with broader systems that access, infer, and reuse user information across components and over time. While such use of user information is integral to personalization, it also raises important privacy questions. In this paper, we argue that individual model- or component-level analyses may not capture all privacy risks arising in such systems, motivating a system-level perspective on privacy. We distinguish and analyze four interconnected privacy-risk channels in personalized AI, and subsequently propose four requirements for system-level privacy evaluation, covering interaction trajectories, internal information flows, indirect leakage, and the privacy-utility trade-off. We argue for their systematic incorporation into privacy audits of personalized AI.

Problem

• Personalized AI
• System-level privacy
• Privacy risk
• Information flow
• Privacy audit

Innovation

• Personalized AI
• System-level Privacy
• Privacy Auditing
• Information Flow
• Privacy-Utility Trade-off