private nuisance estimation

This work addresses three key challenges in differentially private statistical estimation of black-box functions: unknown sensitivity, low query efficiency, and suboptimal data utilization. We propose the first general framework that achieves both statistical and oracle query efficiency without prior knowledge of sensitivity. Our method integrates an adaptive querying mechanism with customized noise injection, guaranteeing ε-differential privacy while drastically reducing dependence on function evaluations—scaling far better than exponential in the problem parameters. We establish a tight theoretical lower bound on the privacy–utility trade-off and prove that our framework attains this bound. Empirical evaluation across multiple benchmark tasks demonstrates that our approach achieves higher estimation accuracy with significantly fewer oracle queries compared to existing differentially private black-box estimators.

In differential privacy, applying nonlinear transformations to noisy statistics (e.g., Laplace-mechanism outputs) induces systematic bias due to the lack of closed-form unbiased estimators. Method: This paper proposes a post-processing-only unbiased estimation framework: (i) it introduces statistical deconvolution to the DP post-processing setting for the first time; (ii) it constructs an unbiased estimator for general twice-differentiable functions; (iii) it designs the first mechanism that yields an unbiased estimate of the private mean even when the sample size is unknown; and (iv) it extends the framework to per-record DP and polynomial function estimation. Contributions/Results: Theoretical analysis guarantees unbiasedness under finite-moment noise. Experiments demonstrate significant improvements over Kamath et al. (2023) in jointly estimating the private mean and sample size, and stronger privacy guarantees than Finley et al. (2024) under per-record DP.

This paper addresses the sequential release of means and variances over multiple mutually exclusive subsets under user-level differential privacy, assuming heterogeneous data and publicly known per-subset user contribution counts. The goal is to maintain fixed statistical estimation error while mitigating the rapid degradation of privacy loss as the number of subsets increases. We propose an iterative algorithm based on user-contribution suppression and, for the first time, derive exact closed-form expressions for the global sensitivity and worst-case bias of mean and variance estimators under truncation/suppression mechanisms. Theoretically, we prove that this mechanism significantly reduces the cumulative privacy budget consumption rate. Empirically, experiments on both real and synthetic datasets demonstrate that, for a fixed estimation error, the privacy loss degradation factor decreases by several-fold; moreover, when the number of users per subset is fixed, the worst-case estimation error is substantially improved.

This paper addresses privacy accounting for subsampling mechanisms—specifically Poisson and without-replacement sampling—in compositional settings under differential privacy (DP), identifying two prevalent misuses: (i) erroneously assuming the worst-case dataset for a single step suffices for adaptive composition analysis, and (ii) conflating the distinct privacy loss characteristics of the two sampling schemes. Method: We rigorously prove that privacy parameters for subsampled composition cannot be derived by naïvely composing single-step worst-case guarantees. Leveraging Rényi differential privacy and exact privacy loss distribution analysis, we develop a numerical accounting framework incorporating counterexample construction and tight theoretical bounds. Contribution/Results: We establish a decidable criterion for detecting and correcting such misuses, and demonstrate—under typical DP-SGD parameters—that ε values for Poisson and without-replacement sampling may differ by over an order of magnitude. Empirical evaluation confirms our framework prevents significant over- or under-estimation of privacy budgets, substantially improving the reliability of privacy guarantees.

This work addresses the excessive noise in differentially private (DP) linear queries—such as sum, mean, and count—caused by high global sensitivity. We propose a noise-reduction method based on simplex projection: mapping high-sensitivity data onto a fixed-norm probability simplex to reuse privacy loss without increasing the ε budget. We first identify a “free lunch” phenomenon for linear queries on the simplex: redundant queries can be answered with zero additional privacy cost. Leveraging sensitivity analysis, decomposition of linear queries, and algebraic reconstruction, we theoretically prove—and empirically validate—that our method reduces the variance of DP estimates by a factor of O(n) under ε-DP, significantly improving accuracy for mean and sum estimation while preserving strict privacy guarantees.

This study addresses the challenge of performing efficient and robust statistical inference for target parameters—such as causal effects—under local differential privacy constraints. We propose a novel integration of the rate-double-robust inference framework with local privacy mechanisms, where noise is injected into individual-level data to ensure privacy protection. Leveraging semiparametric theory, our approach successfully transfers desirable properties of non-private estimators to the privatized setting. The resulting method guarantees unbiasedness and achieves semiparametric efficiency, while also preserving the original estimator’s convergence rate under privacy perturbations. Furthermore, it maintains favorable asymptotic performance under both nonparametric and parametric perturbation regimes, demonstrating broad applicability and robustness in practical privacy-preserving inference tasks.

This work addresses the challenge of simultaneously achieving computational efficiency, statistical optimality, and differential privacy in high-dimensional Bayesian estimation. Focusing on Gaussian mean estimation and linear regression, it proposes the first computationally efficient differentially private algorithm that attains near-Bayes-optimal mean squared error under a Gaussian prior. By introducing a novel framework that translates privacy guarantees into robustness properties and incorporating new constraints based on short-flat decompositions, the authors extend the Sum-of-Squares (SoS) method to construct robust estimators for non-robust objectives such as empirical means and ordinary least squares. The resulting algorithm achieves a $(1+o(1))$ multiplicative factor over the Bayes-optimal error in both settings and is provably computationally optimal within the low-degree polynomial model, substantially outperforming existing efficient approaches.

This work addresses the high sample complexity inherent in estimating monotone statistics under differential privacy. The authors propose an improved subsample-and-aggregate algorithm that introduces a tunable parameter \( t \) to achieve a controllable trade-off between runtime and sample complexity. While maintaining polynomial time complexity, the method reduces the required sample size by a factor of \( t \) compared to conventional approaches and is nearly optimal in terms of query complexity. Empirical evaluations demonstrate that the algorithm substantially enhances performance in private estimation tasks, including eigenvalue estimation, loss estimation, and single-parameter estimation in high-dimensional models.

This work investigates the optimal conversion from Rényi differential privacy (RDP) to $f$-differential privacy ($f$-DP) to derive the tightest possible privacy guarantees. By analyzing the geometric structure of RDP privacy regions—particularly their convexity and the boundary characterized by the Bernoulli mechanism—the authors propose a black-box transformation method based on the intersection of single-order RDP regions. This approach achieves global optimality simultaneously across all RDP profiles and Type I error levels, rigorously establishing for the first time a theoretical limit on the $f$-DP bounds inferable solely from RDP information. The resulting tightest $f$-DP bound is given by the pointwise supremum of individual single-order RDP-induced bounds in function space, thereby unifying and strengthening existing results.

Existing methods for constructing nonparametric confidence intervals under differential privacy rely either on strong distributional assumptions or problem-specific algorithms, limiting their generality and applicability. Method: We propose the first universal black-box framework that constructs asymptotically efficient and tight nonparametric confidence intervals using *any* differentially private estimator. Our approach leverages subsample resampling and privacy-preserving post-processing of the empirical cumulative distribution function, inherently achieving privacy amplification without requiring additional distributional assumptions. Contribution/Results: We establish theoretical guarantees of uniformly asymptotically correct coverage. Empirically, across diverse real-world datasets, our intervals achieve coverage rates consistently near the nominal level, with average widths substantially narrower than those of existing specialized algorithms—and statistical efficiency approaching that of the non-private oracle benchmark.