Let's Ask Gauss: Improved One-Run Privacy Auditing

2026-06-10

AI Summary

This work addresses the limitation of existing single-run differential privacy (DP) auditing methods, which yield loose lower bounds on privacy leakage due to information loss from binarizing sentinel signals. Focusing on DP machine learning algorithms such as DP-SGD, the paper proposes an efficient single-run auditing framework that leverages the distributional properties of sequences of sentinel signals. It establishes, for the first time, that normalized aligned sentinel signals asymptotically follow a Gaussian distribution, and builds upon this insight a Gaussian approximation auditing method grounded in the Central Limit Theorem. Requiring only a single training run, the proposed approach achieves substantially tighter lower bounds on privacy leakage compared to current techniques, thereby significantly enhancing both the accuracy and practicality of DP auditing.

Abstract

Privacy auditing provides an important safeguard by estimating the actual information leaked by a model, thus ensuring that theoretical privacy guarantees hold in practice. We study empirical privacy auditing for differentially private (DP) machine learning, focusing on efficient one-run methods for mechanisms such as DP-SGD. Prior one-run approaches threshold training examples or "canaries" into binary membership guesses, which discards useful information. We show that, in the white-box DP-SGD setting, canary-aligned signals naturally form a sequence of random variables whose normalized sum is asymptotically Gaussian. Leveraging this distributional perspective, we develop a DP-auditing framework that leads to tighter privacy lower bounds from a single training run.

Problem

• privacy auditing
• differential privacy
• DP-SGD
• one-run methods
• membership inference

Innovation

• privacy auditing
• differential privacy
• DP-SGD
• Gaussian approximation
• one-run method