privacy accounting

Quantitatively tracking and composing privacy loss under formal definitions (e.g., differential privacy or zCDP), deriving tight composition bounds, and allocating limited privacy budget to tasks like synthetic data generation to maximize utility while preserving guarantees.

privacyaccounting

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

This study addresses the challenge of setting the privacy budget ε in differential privacy, which often lacks intuitive grounding and complicates the trade-off between privacy protection and data utility. The authors propose an interpretable method for calibrating ε by imposing a strict upper bound on an adversary’s posterior odds of inferring sensitive information, directly translating this constraint into a corresponding ε value. Grounded in differential privacy theory, Bayesian inference, and posterior odds analysis, the approach is empirically validated using case-control study data. Results demonstrate that the framework effectively maps real-world privacy requirements to concrete ε values, achieving rigorous privacy guarantees while preserving analytical accuracy, thereby significantly enhancing the practicality and operational feasibility of ε selection.

adversarydifferential privacyepsilon

Interpreting Differential Privacy in Terms of Disclosure Risk

Jul 13, 2025
ZK
Zeki Kazan
🏛️ Duke University | TikTok Inc

Differential privacy (DP) lacks intuitive interpretability from the perspective of statistical disclosure risk, hindering practitioners’ understanding and trustworthy deployment. Method: This work establishes, for the first time, a rigorous theoretical linkage between DP parameters (ε, δ) and quantifiable disclosure risk. By integrating statistical inference theory, privacy analysis, and risk modeling, we derive tight upper bounds on an adversary’s worst-case success probability in inferring sensitive attributes under DP. Contribution/Results: The framework endows ε and δ with concrete, risk-based semantics—interpreting them as guarantees on bounded inference risk. It further provides a risk-accumulation interpretation of composition theorems, enabling principled, scenario-aware selection and validation of privacy parameters. Our results significantly enhance the interpretability and credibility of DP, offering both theoretical foundations and actionable guidelines for privacy engineering practice.

Develop tools for reasoning about privacy guaranteesInterpret differential privacy in terms of disclosure riskLink DP to statistical disclosure risk measures

Synthetic Data: Revisiting the Privacy-Utility Trade-off

Jul 09, 2024
FJ
F. J. Sarmin
🏛️ University of Manitoba | Concordia University

Recent critiques have challenged the differential privacy guarantees of PATE-GAN and PrivBayes, questioning the validity of their privacy-utility trade-offs. However, these critiques rely on restrictive assumptions—such as synthetic or simplistic data distributions—and limited experimental settings, potentially biasing their conclusions. Method: We propose a more general privacy-utility evaluation framework that integrates privacy game analysis and theoretical verification, and conduct k-anonymity benchmarking experiments on real-world datasets without distributional assumptions. Contribution/Results: Under identical privacy budgets, both PATE-GAN and PrivBayes significantly outperform k-anonymity in statistical utility while maintaining strong differential privacy guarantees. We demonstrate that prior claims of “privacy failure” stem from flawed evaluation premises—specifically, the absence of rigorous privacy accounting and realistic data assumptions. Our empirical analysis refutes these criticisms and establishes synthetic data generation as a robust and effective privacy-enhancing technology.

Challenges synthetic data's privacy-utility trade-off superiority.Identifies breaches in differential privacy guarantees of PATE-GAN and PrivBayes.Reaffirms synthetic data's favorable privacy-utility trade-off in general environments.

Avoiding Pitfalls for Privacy Accounting of Subsampled Mechanisms under Composition

May 27, 2024
CL
C. Lebeda
🏛️ IT University of Copenhagen | University of Waterloo | Vector Institute | Google DeepMind

This paper addresses privacy accounting for subsampling mechanisms—specifically Poisson and without-replacement sampling—in compositional settings under differential privacy (DP), identifying two prevalent misuses: (i) erroneously assuming the worst-case dataset for a single step suffices for adaptive composition analysis, and (ii) conflating the distinct privacy loss characteristics of the two sampling schemes. Method: We rigorously prove that privacy parameters for subsampled composition cannot be derived by naïvely composing single-step worst-case guarantees. Leveraging Rényi differential privacy and exact privacy loss distribution analysis, we develop a numerical accounting framework incorporating counterexample construction and tight theoretical bounds. Contribution/Results: We establish a decidable criterion for detecting and correcting such misuses, and demonstrate—under typical DP-SGD parameters—that ε values for Poisson and without-replacement sampling may differ by over an order of magnitude. Empirical evaluation confirms our framework prevents significant over- or under-estimation of privacy budgets, substantially improving the reliability of privacy guarantees.

Clarifying misconceptions about worst-case dataset assumptions in compositionComparing privacy differences between Poisson and without-replacement samplingComputing tight privacy guarantees for composed subsampled mechanisms

Concurrent Composition for Interactive Differential Privacy with Adaptive Privacy-Loss Parameters

Sep 12, 2023
SH
Samuel Haney
🏛️ Tumult Labs | Harvard University

This work addresses the composition of interactive differential privacy (DP) mechanisms under *adaptive selection of privacy-loss parameters* and *concurrent invocation*, including interleaved queries and dynamically instantiated mechanisms. To overcome the limitation of existing theory—which fails to model adaptive, concurrent adversarial interaction—the paper establishes, for the first time, that *privacy filters* and *privacy odometers*, originally designed for non-interactive settings, admit rigorous generalization to concurrent interactive settings. This result unifies and proves the robustness of $(varepsilon,delta)$-DP, $f$-DP, and fixed-order Rényi DP under concurrent composition: concurrency itself does not degrade privacy guarantees. The paper develops the first theoretical framework for concurrent interactive DP supporting fully adaptive privacy budget management, and releases an open-source, production-ready implementation—providing both foundational guarantees and practical engineering support for real-world private systems.

Extend privacy filters and odometers to interactive mechanisms under various DP measuresProvide implementation for deploying fully adaptive differentially private interactive mechanismsStudy concurrent composition of interactive mechanisms with adaptive privacy-loss parameters

Latest Papers

What's happening recently
View more

Composition Theorems for f-Differential Privacy

Dec 23, 2025
NF
Natasha Fernandes
🏛️ Macquarie University | UNSW

This work addresses the looseness of privacy loss estimation under $f$-differential privacy (fDP) in complex composition scenarios. We propose a novel compositional analysis framework grounded in statistical hypothesis testing and quantitative information flow (QIF) channel models. First, we establish a Galois connection between fDP and QIF, revealing their fundamental equivalence under the hypothesis-testing semantics. Leveraging this connection, we derive a general composition theorem that is both broadly applicable and significantly tighter than existing bounds. Our method substantially improves the precision of compositional analysis for canonical mechanisms—particularly the Gaussian mechanism—yielding more accurate and computationally tractable privacy loss predictions. By bridging theoretical fDP characterizations with practical budget management, this framework advances the operational deployment of fDP in real-world privacy-preserving systems.

Enables improved analysis of complex privacy designsEstablishes equivalence between fDP and Quantitative Information FlowProves composition theorems for f-differential privacy mechanisms

How to Get Actual Privacy and Utility from Privacy Models: the k-Anonymity and Differential Privacy Families

Oct 13, 2025
JD
Josep Domingo-Ferrer
🏛️ Universitat Rovira i Virgili | CYBERCAT-Center for Cybersecurity Research of Catalonia

Existing mainstream privacy models suffer from fundamental limitations: k-anonymity operates syntactically, rendering it vulnerable to background knowledge attacks and lacking semantic constraints; differential privacy faces a sharp utility–privacy trade-off—small privacy budgets cause severe data distortion, while large budgets degrade privacy guarantees. Method: We propose Semantic k-Anonymity, which formally incorporates domain-specific semantic constraints and dependencies among sensitive attributes to reconstruct the equivalence-class partitioning mechanism—enhancing disclosure resistance without compromising data utility. Contribution/Results: Through rigorous formal modeling, principled semantic constraint design, and empirical risk assessment, we demonstrate that Semantic k-Anonymity achieves more robust privacy protection and higher data utility than conventional k-anonymity and differential privacy in realistic settings, thereby reducing reliance on post-hoc risk evaluation.

Current privacy models require costly empirical risk assessmentsDifferential privacy causes excessive utility loss with small budgetsk-anonymity fails to fully prevent data disclosure risks

This work addresses the challenge non-technical users face in understanding the trade-off between privacy loss parameters and the reliability of statistical inference in differential privacy. By reframing the privacy–utility trade-off within a hypothesis testing framework, the paper introduces the concept of “relative disclosure risk,” which directly links privacy loss parameters to the validity of statistical inference. Through theoretical analysis of how differentially private mechanisms affect the significance of hypothesis tests, the study quantifies the uncertainty introduced by randomization and its effectiveness in mitigating membership inference attacks. The resulting insights yield an actionable guideline for non-expert users to select appropriate privacy mechanisms, thereby promoting transparent deployment and principled configuration of differential privacy in practical applications.

data analysisdifferential privacydisclosure risk

This work addresses the challenge of precisely characterizing the overall privacy guarantee when composing mechanisms under multiple heterogeneous differential privacy (DP) constraints. The authors propose a general composition framework that, for the first time, enables an exact description of the resulting privacy region after composing an arbitrary number of mechanisms subject to diverse DP bounds. By constructing a binary hypothesis testing–based mixture model and integrating probabilistic mixing with f-DP approximation techniques, the framework yields an exact composition theorem for multiple DP constraints. Moreover, the approach naturally extends to the f-DP setting, significantly enhancing both the tightness and applicability of compositional privacy analysis.

compositiondifferential privacyf-DP

This work addresses the high computational cost and poor scalability of existing optimization-based metric differential privacy (mDP) mechanisms in continuous, fine-grained domains, which stem from the need to construct dense perturbation matrices and enforce pointwise constraints. To overcome these limitations, we propose the first framework that integrates interpolation with mDP: perturbation distributions are optimized only at sparse anchor points, and distributions at non-anchor locations are generated via one-dimensional stepwise decomposition and log-convex interpolation. A correction formula is introduced to rigorously satisfy ℓp-norm mDP constraints, while jointly optimizing both the perturbation distributions and the allocation of privacy budgets across dimensions. Experiments on real-world location datasets demonstrate that our approach achieves strong privacy guarantees without sacrificing utility, significantly outperforming existing baselines in fine-grained domains.

continuous domainsfine-grained domainslp-norm

Hot Scholars

FJ

Frederik J. Zuiderveen Borgesius

Professor ICT and Law, iHub, Radboud University, The Netherlands
Law and technologyprivacydata protection lawnon-discrimination law
SZ

Sebastian Zimmeck

Associate Professor of Computer Science, Wesleyan University
Internet PrivacyPrivacy-enhancing TechnologiesGlobal Privacy ControlPrivacy Law
KK

Konrad Kollnig

Assist. Professor (PhD Oxford), Maastricht University
Regulatory TechnologiesPrivacyAI Law
SP

Shidong Pan

Postdoctoral Researcher of New York University & Columbia University
Usable Privacy and SecurityPrivacy PolicyResponsible AISoftware Engineering