Differential Privacy of Gradient Descent on Perturbed Objectives

2026-10-01

AI Summary

This study addresses the explicit dimension dependence in existing privacy bounds for generalized linear models under objective perturbation with finite-step gradient descent. The proposed method builds upon deterministic gradient descent augmented with Gaussian noise perturbation, employing diffeomorphic mappings for variable substitution and leveraging convex optimization theory for rigorous analysis. Under strongly convex and smooth conditions, this work eliminates the explicit dimensional factor from the privacy bound. Furthermore, it demonstrates that the excess empirical risk converges at a geometric rate, and the correction term generalizes to population risk without requiring an additional condition number multiplier. Consequently, the framework achieves a dimension-independent privacy-utility tradeoff.

Abstract

Objective perturbation adds a random linear term to a regularized empirical risk and releases the exact perturbed minimizer. We study the finite computation obtained by releasing the $N$-th iterate of deterministic gradient descent on $w\mapsto F(w;S)+\langle z,w\rangle$, where $z\sim\mathcal N(0,σ^2I_d)$ is drawn once before optimization. For strongly convex and smooth objectives with Lipschitz Hessian, we prove an explicit condition under which the map $z\mapsto w_N$ is a $C^1$-diffeomorphism on the bounded domains used in the privacy argument, with a quantitative lower bound on the smallest singular value of its Jacobian. This permits a direct change-of-variables analysis of the finite iterate. For generalized linear models, the resulting privacy-profile bound has no explicit ambient-dimension factor once the iteration condition holds, and its finite-iteration correction decreases geometrically. By letting the free truncation parameter grow slowly with $N$, we recover the corresponding exact-minimizer certificate in the limit. We also bound the expected excess empirical risk by $dσ^2/(2μ)$ plus a geometrically decreasing optimization term, and transfer the result to population risk without an additional multiplicative condition-number factor in the leading statistical terms.

Problem

• Differential Privacy
• Objective Perturbation
• Gradient Descent
• Finite Computation
• Excess Risk

Innovation

• Differential Privacy
• Objective Perturbation
• Gradient Descent
• Change-of-Variables
• Generalized Linear Models