dithered gaussian mechanism

This work addresses critical limitations of the conventional discrete Gaussian mechanism in differential privacy, which is vulnerable to floating-point precision issues and demands substantial high-quality randomness. The authors propose the dithered Gaussian mechanism, which decouples randomness into a privacy-critical high-quality component and a non-critical, computationally efficient component through output-side discretization and dual-source randomization. This approach preserves the theoretical privacy guarantees of the standard Gaussian mechanism while eliminating floating-point security vulnerabilities. Notably, it drastically reduces the requirement for high-quality random bits—rendering this demand independent of noise magnitude—and enables cryptographically secure noise generation in DP-SGD with minimal computational overhead, thereby achieving a strong balance between security and practicality.

Existing differentially private (DP) noise mechanisms—particularly the Gaussian mechanism—lack a principled theoretical foundation for why β = 2 is empirically optimal in frameworks like PATE and DP-SGD. Method: This work systematically investigates the Generalized Gaussian (GG) mechanism (with shape parameter β ∈ [1, 2]) for privacy-preserving machine learning. We formally prove that the entire GG family satisfies (ε, δ)-differential privacy, and introduce a dimension-agnostic Privacy Random Variable (PRV)-based accounting framework that reduces privacy loss computation complexity from O(d) to O(1). Contribution/Results: Our theoretical analysis shows that tuning β yields only marginal utility gains, explaining the empirical dominance of the Gaussian mechanism (β = 2). Extensive experiments confirm that β ≈ 2 achieves the optimal trade-off between model accuracy and privacy budget consumption. The work provides a unified theoretical framework and empirical validation for selecting DP noise mechanisms.

To address the fundamental trade-off between privacy preservation and utility in high-dimensional, multi-feature data, this paper proposes a coordinate-wise independent but non-identically distributed (i.n.i.d.) noise mechanism. We provide the first theoretical proof that the Laplace mechanism can strictly outperform the Gaussian mechanism in high dimensions under differential privacy. By explicitly modeling heterogeneous privacy sensitivities across coordinates, we derive formal privacy guarantees for i.n.i.d. noise and establish an optimization framework for designing optimal noise parameters—supporting both weighted mean-squared error (MSE) and ℓₚ norm error minimization. The method is instantiated in private coordinate descent, differentially private PCA, and private deep learning with group-wise gradient clipping. Empirical evaluation demonstrates consistent utility gains over state-of-the-art baselines under identical privacy budgets across optimization, dimensionality reduction, and model training tasks—challenging the conventional wisdom that Gaussian noise is inherently superior in high-dimensional settings.

This paper addresses privacy accounting for subsampling mechanisms—specifically Poisson and without-replacement sampling—in compositional settings under differential privacy (DP), identifying two prevalent misuses: (i) erroneously assuming the worst-case dataset for a single step suffices for adaptive composition analysis, and (ii) conflating the distinct privacy loss characteristics of the two sampling schemes. Method: We rigorously prove that privacy parameters for subsampled composition cannot be derived by naïvely composing single-step worst-case guarantees. Leveraging Rényi differential privacy and exact privacy loss distribution analysis, we develop a numerical accounting framework incorporating counterexample construction and tight theoretical bounds. Contribution/Results: We establish a decidable criterion for detecting and correcting such misuses, and demonstrate—under typical DP-SGD parameters—that ε values for Poisson and without-replacement sampling may differ by over an order of magnitude. Empirical evaluation confirms our framework prevents significant over- or under-estimation of privacy budgets, substantially improving the reliability of privacy guarantees.

This paper addresses the $d$-dimensional counting query problem under differential privacy with add/remove neighborhood relations. Conventional independent Gaussian mechanisms incur a per-query standard deviation of $sqrt{d}$, constrained by the fundamental variance lower bound. We propose a structure-aware Gaussian mechanism that jointly designs globally correlated and independent Gaussian noise, explicitly modeling the covariance matrix to capture the intrinsic geometric structure of the sensitivity space. This design reduces the per-query standard deviation to $(sqrt{d}+1)/2$, breaking the theoretical limitation of independent-noise mechanisms. Theoretical analysis establishes the mechanism’s generality, showing direct applicability to other multidimensional query tasks sharing similar sensitivity structures. Extensive experiments demonstrate significant improvements in total noise standard deviation over state-of-the-art baselines, achieving both rigorous $(varepsilon,delta)$-differential privacy guarantees and substantially enhanced statistical utility.

In the moderate-to-low privacy regime (i.e., small $(\varepsilon, \delta)$), existing Gaussian mechanisms are significantly suboptimal due to excessive noise injection. This work proposes a hybrid Gaussian noise mechanism that constructs a convex combination of multiple Gaussian distributions with identical variances but distinct means, adaptively tuning both the means and mixing weights using sensitivity information. It presents the first systematic construction and analysis of a Gaussian mixture-based perturbation scheme satisfying $(\varepsilon, \delta)$-differential privacy. The authors derive tight variance conditions and an efficient algorithm that substantially reduce both L1 and L2 utility loss in the low-privacy regime, markedly narrowing the performance gap with the theoretically optimal mechanism and achieving near-optimal accuracy.

This work investigates privacy leakage arising from releasing posterior sample paths of Gaussian processes under the strict setting where training data are entirely private. It establishes, for the first time, that the inherent randomness of posterior sampling naturally provides differential privacy guarantees, and derives rigorous privacy bounds using Rényi differential privacy theory. The study proposes effective ridge regularization as a core mechanism to control privacy levels, complemented by calibrated noise injection for enhanced protection. Both theoretical analysis and empirical results demonstrate that the degree of privacy leakage is significantly influenced by the strength of regularization, posterior variance, and the number of released samples. In settings with noisy observations, moderate regularization effectively safeguards privacy while preserving utility for downstream tasks.

This work addresses the long-standing trade-off between privacy and utility in additive noise mechanisms for high-dimensional real-valued vector queries under differential privacy, where no universally optimal solution has previously existed. The study establishes, for the first time, the asymptotic optimality of the Gaussian mechanism in high dimensions and introduces a novel family of spherical generalized gamma mechanisms that significantly outperform existing approaches in low-dimensional settings. This mechanism family unifies the Gaussian and ℓ₂ mechanisms within a single framework and resolves the open problem of tight composition bounds for the ℓ₂ mechanism. Consequently, the authors derive tight privacy composition bounds applicable across both low- and high-dimensional regimes.

This work addresses a central challenge in differential privacy: enhancing algorithmic utility without compromising privacy guarantees or incurring excessive computational complexity. The authors propose a post-processing denoising method grounded in empirical Bayes estimation, which effectively reduces mean squared error using only the outputs of Gaussian differential privacy mechanisms. To the best of our knowledge, this is the first systematic application of the empirical Bayes framework to differential privacy post-processing. The approach significantly improves utility without altering the underlying privacy mechanism, offering both simplicity and broad applicability. Empirical evaluations demonstrate consistent performance gains over existing differentially private algorithms across diverse tasks, including histogram release, principal component analysis, and linear regression.

本文通过使用IBM的DiffPrivLib系统性地研究了随机数质量下降对差分隐私机制的影响，揭示了不同熵源下隐私损失的变化情况。