gmm estimation

This paper addresses the problem of efficiently estimating a single parameter in parametric models under differential privacy. Existing methods suffer from limitations in computational and statistical efficiency, as well as suboptimal accuracy bounds. To overcome these, we propose a novel framework grounded in the stability of local estimators—marking the first approach that enables self-generation and self-verification of private stability certificates. Our framework supports adaptive privacy mechanism design and achieves asymptotically instance-optimal error bounds, surpassing the accuracy limits of generic mechanisms such as Laplace noise injection or DP-SGD. Theoretical analysis is complemented by Monte Carlo simulations and empirical evaluation on real-world ACS and Census datasets. Results demonstrate that our method attains the optimal convergence rate and significantly outperforms baselines—especially in small-sample and high-dimensional constrained settings—thereby establishing a new paradigm for practical, high-accuracy differentially private parameter estimation.

This paper addresses the challenge of differentially private (DP) statistical estimation for unbounded-support data. We propose the first generic DP estimation framework based on systematic data truncation, applicable to high-dimensional exponential families—including Gaussian mean and covariance estimation. Unlike conventional approaches relying on problem-specific sensitivity analysis, our method systematically incorporates truncation statistics into DP estimation, augmented with a bias-correction mechanism and an improved uniform convergence bound for the truncated log-likelihood, effectively mitigating truncation-induced bias. The algorithm integrates data truncation, maximum likelihood estimation, and DP stochastic gradient descent, achieving computational efficiency alongside near-optimal sample complexity. Experiments demonstrate state-of-the-art privacy–accuracy trade-offs for Gaussian mean and covariance estimation. Our framework establishes a scalable new paradigm for DP statistical modeling over unbounded data.

This work addresses robust linear regression under heavy-tailed noise while preserving differential privacy. The authors propose an estimation framework that integrates a tunable Huber loss with a differentially private mechanism. In low-dimensional settings, they employ noisy truncated gradient descent, and in high-dimensional sparse regimes, they adopt a noisy iterative hard thresholding algorithm, achieving a balance among privacy, robustness, and statistical efficiency. Theoretical analysis precisely characterizes the non-asymptotic convergence rates, revealing their dependence on the moment exponent, privacy parameters, sample size, and intrinsic dimensionality, and elucidating the trade-offs among bias, privacy, and robustness. The method attains near-optimal rates under sub-Gaussian errors and demonstrates strong empirical performance under heavy-tailed noise, as validated by both theoretical guarantees and experiments on synthetic data and two real-world datasets.

Under existing differential privacy (DP) frameworks, there is a lack of general-purpose statistical inference methods—particularly when privately releasing multiple bootstrap estimates to construct confidence intervals (CIs), where privacy cost accumulation remains intractable and theoretical guarantees for sampling distribution inference are absent. This paper introduces DP Bootstrap, a novel paradigm: (i) it establishes the first universal privacy cost analysis for a single DP bootstrap release; (ii) it proposes a numerical composition method to precisely aggregate privacy budgets across multiple releases; (iii) it achieves asymptotically optimal privacy guarantees within the Gaussian DP (GDP) framework; and (iv) it pioneers DP inference for quantile regression. Theoretically, the resulting CIs attain nominal coverage, while point estimators enjoy consistency, asymptotic efficiency, and minimax-optimal convergence rates. Empirical evaluation on the 2016 Canadian Census data demonstrates significant improvements over baselines in mean estimation, logistic regression, and quantile regression.

This paper addresses the problem of robustly recovering a one-dimensional probability distribution from noisy Chebyshev moments. For recoverability under the Wasserstein-1 distance, we propose the first differentially private synthetic data algorithm based on linear queries, improving the error bound from the classical $O(1/sqrt{n})$ to the optimal $ ilde{O}(1/n)$ (where $n$ is the number of data points), thereby breaking a long-standing bottleneck. We also establish a tighter theoretical bound for Chebyshev moment matching. Our method integrates Chebyshev polynomial approximation, Wasserstein-distance analysis, differential privacy mechanisms, and spectral theory of random matrices. The results enable high-accuracy, computationally efficient differentially private synthetic data generation and significantly accelerate spectral density estimation for symmetric matrices—outperforming the state-of-the-art STOC’22 approach in both accuracy and efficiency.

Differential privacy typically requires bounded data, posing challenges for handling unbounded datasets. This work proposes Public-moment-guided Truncation (PMT), a novel method that leverages second-order moment information from a small amount of public data to adaptively transform and truncate private data. The truncation radius in PMT depends only on non-sensitive parameters such as dimensionality and sample size. This approach substantially improves the condition number of the data, enhances robustness to noise, and supports mapping back to the original space. Theoretical analysis demonstrates that PMT achieves tighter error bounds, better convergence, and stronger robustness compared to existing methods. Experiments on both synthetic and real-world datasets confirm that PMT significantly improves model accuracy and stability while preserving differential privacy.

This study addresses the challenge of performing efficient and robust statistical inference for target parameters—such as causal effects—under local differential privacy constraints. We propose a novel integration of the rate-double-robust inference framework with local privacy mechanisms, where noise is injected into individual-level data to ensure privacy protection. Leveraging semiparametric theory, our approach successfully transfers desirable properties of non-private estimators to the privatized setting. The resulting method guarantees unbiasedness and achieves semiparametric efficiency, while also preserving the original estimator’s convergence rate under privacy perturbations. Furthermore, it maintains favorable asymptotic performance under both nonparametric and parametric perturbation regimes, demonstrating broad applicability and robustness in practical privacy-preserving inference tasks.

This work addresses the challenge of simultaneously achieving computational efficiency, statistical optimality, and differential privacy in high-dimensional Bayesian estimation. Focusing on Gaussian mean estimation and linear regression, it proposes the first computationally efficient differentially private algorithm that attains near-Bayes-optimal mean squared error under a Gaussian prior. By introducing a novel framework that translates privacy guarantees into robustness properties and incorporating new constraints based on short-flat decompositions, the authors extend the Sum-of-Squares (SoS) method to construct robust estimators for non-robust objectives such as empirical means and ordinary least squares. The resulting algorithm achieves a $(1+o(1))$ multiplicative factor over the Bayes-optimal error in both settings and is provably computationally optimal within the low-degree polynomial model, substantially outperforming existing efficient approaches.

This work addresses the challenge of robust mean estimation under outliers and heavy-tailed distributions while guaranteeing zero-concentrated differential privacy (zCDP). The authors propose a novel method termed “balloon mean,” which integrates iterative clipping with an expanded Mahalanobis-distance ball—referred to as a “balloon”—to achieve both robustness and computational feasibility. Designed for the contaminated ellipsoidal model, the approach requires only a small number of interpretable hyperparameters. Theoretical analysis establishes its statistical optimality and robustness in the presence of heavy-tailed noise and data contamination. Empirical evaluations demonstrate that the balloon mean significantly outperforms existing differentially private mean estimators across a variety of contamination scenarios.

This study addresses the challenge that noise introduced by differential privacy severely impedes effective statistical inference on large-scale privacy-preserving data, as existing approaches often rely on strong parametric assumptions or lack scalability. We propose a two-step approximate Bayesian inference framework: first imputing the differentially private data, then sampling from the non-private posterior distribution. The method achieves asymptotic validity under weak assumptions in large samples while simultaneously satisfying conservative frequentist properties, thereby combining Bayesian flexibility with frequentist reliability. Building upon and refining the approach of Guha and Reiter (2025), we demonstrate the method’s effectiveness and practical utility through simulation studies and an analysis of homeownership using the 2022 American Community Survey.