🤖 AI Summary
This study addresses the complexities of converting between differential privacy notions and the information loss induced by parameter compression. We unify mainstream privacy definitions within an information-theoretic framework by establishing strict equivalence classes among privacy profiles, hypothesis testing, and Rényi Differential Privacy (RDP) curves. This approach precisely delineates conversion boundaries across different privacy concepts and quantifies the accuracy degradation caused by zero-concentrated differential privacy (zCDP) compression. Our results demonstrate that retaining the complete RDP curve reduces noise variance by 45% and improves test accuracy by 8.73 percentage points on the Fashion-MNIST benchmark. Ultimately, this work provides a unified paradigm for the theoretical analysis and optimization of privacy mechanisms.
📝 Abstract
Differential privacy (DP) admits several notions, but the choice among them may affect both privacy analysis and utility. In this paper, we consider four mainstream curve-based privacy notions within a unified information-theoretic framework. For a fixed ordered pair of output distributions, we establish information equivalence among the two directional privacy profiles of $(\varepsilon,δ)$-DP, the pair of hypothesis-testing trade-off functions, and the extended privacy-loss distribution. The exact Rényi differential privacy (RDP) curve joins this equivalence class whenever it is finite at some order greater than one. Under this mild condition, choosing among these notions changes only their semantic interpretation and computational requirements. In contrast, taking the maximum of the directional privacy profiles or compressing the RDP curve into a single zero-concentrated differential privacy (zCDP) parameter can lose information. We quantify the information loss between the exact RDP curve and its zCDP bound for standard noise mechanisms. This gap is zero for Gaussian noise but generally positive for Gaussian-mixture, Laplace, discrete Gaussian, and Poisson-subsampled Gaussian mechanisms. Moreover, this gap grows linearly with the number of independently composed mechanisms. Our information-theoretic perspective has practical consequences. At the same certified privacy level, retaining the full RDP curve rather than using zCDP reduces the required noise variance by up to $45\%$ for Gaussian-mixture noise in workloads comparable in size to the American Community Survey. For DP-SGD on Fashion-MNIST under Poisson subsampling, an RDP-based privacy accountant improves test accuracy by up to $8.73$ percentage points compared to a zCDP-based accountant when both are calibrated to the same $(\varepsilon,δ)$ guarantee.