Differentially Private Nonparametric Modal Learning with Applications to Regression and Clustering

2026-07-31

AI Summary

This work addresses the problem of nonparametric mode estimation for multimodal distributions under strict differential privacy constraints, proposing a novel method named DP-GRAMS. Built upon the mean-shift framework, DP-GRAMS integrates higher-order kernel density estimation, gradient clipping, and a calibrated Gaussian mechanism to recover modes via noisy gradient ascent while preserving privacy. It further incorporates density-aware initialization and a multi-start joint release strategy to enhance utility. Theoretical analysis demonstrates that DP-GRAMS recovers all population modes with high probability and achieves a near-minimax optimal error rate. Empirical evaluations on both synthetic and real-world datasets show that DP-GRAMS significantly outperforms existing baselines and naturally extends to regression and clustering tasks.

Abstract

Density modes provide a localized and interpretable summary of multimodal distributions, but their estimation under rigorous differential privacy constraints remains largely unexplored. We study differentially private recovery of density modes for multivariate distributions under local smoothness, curvature, and separation conditions. We propose DP-GRAMS, a mean-shift inspired method that performs noisy ascent on a differentially private score estimator. Assuming the density belongs locally to a Hölder class with smoothness parameter $β> 2$, our score estimator uses bias-reducing higher-order kernels, and then enforces privacy in the gradient ascent steps via gradient clipping and calibrated Gaussian noise. A private initialization scheme combines a density-aware utility with a suppression rule and, with $k\asymp M\log n$ draws over a public $h_{\mathrm{DAP}}$-grid and suppression radius $ρ_{\mathrm{init}}\asymp (\log n)^{-1/d}$, achieves high-probability coverage of the modal basins by successively suppressing selected local neighborhoods in competitive regions, while correlated noise across multiple starts enables joint release under a single $(\varepsilon,δ)$-differential privacy guarantee. We prove that all population modes are recovered with high probability and establish asymptotic error rates of the form $O\!\left((\tfrac{\log n}{n})^{\frac{2(β-1)}{d+2β}}\right) + O\!\left((\tfrac{\mathrm{polylog}(n,δ)}{n^2\varepsilon^2})^{\frac{β-1}{d+β}}\right)$. We also provide minimax lower bounds for private mode estimation, and show that our estimators are nearly optimal, up to a logarithmic factor in the MSE. We present two natural extensions: DP-PMS, a private modal-regression method, and DP-GRAMS-C, a clustering pipeline. Extensive experiments on synthetic and real data demonstrate favorable privacy-utility trade-offs relative to common baselines.

Problem

• Differential Privacy
• Density Modes
• Nonparametric Learning
• Multivariate Distributions
• Mode Estimation

Innovation

• Differentially Private Mode Estimation
• Nonparametric Density Modes
• DP-GRAMS
• Higher-order Kernel Smoothing
• Private Initialization with Suppression