Privacy Loss of Noise Perturbation via Concentration Analysis of A Product Measure

2025-12-05

AI Summary

This paper addresses the problem of excessive noise and low utility in $(varepsilon,delta)$-differential privacy (DP) mechanisms for high-dimensional data. To tackle this, we propose a novel perturbation framework leveraging the geometric properties of spherically symmetric noise. Our method models the noise distribution as a product measure $W imes U$, where the radial component $W$ and the uniform directional component $U$ are independent; this enables a geometric characterization of the privacy loss random variable and yields closed-form moment bounds via concentration analysis—rigorously guaranteeing $(varepsilon,delta)$-DP. Compared to the classical Gaussian mechanism, our approach significantly reduces the expected noise magnitude under the same privacy budget, with theoretical analysis showing slower growth of the expected norm with dimensionality. Experiments demonstrate substantial improvements in utility and accuracy for high-dimensional tasks such as empirical risk minimization (ERM), enhancing the practicality of private data release.

Technology Category

• Machine Learning: Privacy
• Game Theory and Economic Paradigms: Mechanism Design
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Noise perturbation is one of the most fundamental approaches for achieving $(ε,δ)$-differential privacy (DP) guarantees when releasing the result of a query or function $f(cdot)inmathbb{R}^M$ evaluated on a sensitive dataset $mathbf{x}$. In this approach, calibrated noise $mathbf{n}inmathbb{R}^M$ is used to obscure the difference vector $f(mathbf{x})-f(mathbf{x}')$, where $mathbf{x}'$ is known as a neighboring dataset. A DP guarantee is obtained by studying the tail probability bound of a privacy loss random variable (PLRV), defined as the Radon-Nikodym derivative between two distributions. When $mathbf{n}$ follows a multivariate Gaussian distribution, the PLRV is characterized as a specific univariate Gaussian. In this paper, we propose a novel scheme to generate $mathbf{n}$ by leveraging the fact that the perturbation noise is typically spherically symmetric (i.e., the distribution is rotationally invariant around the origin). The new noise generation scheme allows us to investigate the privacy loss from a geometric perspective and express the resulting PLRV using a product measure, $W imes U$; measure $W$ is related to a radius random variable controlling the magnitude of $mathbf{n}$, while measure $U$ involves a directional random variable governing the angle between $mathbf{n}$ and the difference $f(mathbf{x})-f(mathbf{x}')$. We derive a closed-form moment bound on the product measure to prove $(ε,δ)$-DP. Under the same $(ε,δ)$-DP guarantee, our mechanism yields a smaller expected noise magnitude than the classic Gaussian noise in high dimensions, thereby significantly improving the utility of the noisy result $f(mathbf{x})+mathbf{n}$. To validate this, we consider convex and non-convex empirical risk minimization (ERM) problems in high dimensional space and apply the proposed product noise to achieve privacy.

Problem

• Analyzes privacy loss in differential privacy via product measure concentration.
• Proposes spherical symmetric noise generation for improved utility in high dimensions.
• Derives closed-form moment bounds to prove differential privacy guarantees.

Innovation

• Uses product measure W×U for noise generation
• Derives closed-form moment bound for DP proof
• Reduces expected noise magnitude in high dimensions