Differential Privacy of Gaussian Process Posterior Sampling

2026-06-16

AI Summary

This work investigates privacy leakage arising from releasing posterior sample paths of Gaussian processes under the strict setting where training data are entirely private. It establishes, for the first time, that the inherent randomness of posterior sampling naturally provides differential privacy guarantees, and derives rigorous privacy bounds using Rényi differential privacy theory. The study proposes effective ridge regularization as a core mechanism to control privacy levels, complemented by calibrated noise injection for enhanced protection. Both theoretical analysis and empirical results demonstrate that the degree of privacy leakage is significantly influenced by the strength of regularization, posterior variance, and the number of released samples. In settings with noisy observations, moderate regularization effectively safeguards privacy while preserving utility for downstream tasks.

Abstract

We study the privacy of releasing posterior sample paths from a Gaussian process (GP) when the entire training set including covariates and responses is private. Unlike standard differential-privacy (DP) mechanisms that add external noise, posterior sampling is random by construction. We show that this intrinsic randomness yields DP guarantees by deriving explicit Rényi-DP bounds for GP posterior sample-path release. The bounds separate posterior-mean leakage from data-dependent posterior-covariance leakage showing that meaningful privacy depends sharply on effective ridge regularisation. We apply membership-inference attacks to show that empirical leakage follows the predicted dependence on regularisation, posterior variance and the number of released posterior sample-paths. Utility experiments on downstream posterior-sampling tasks identify noisy-observation regimes where privacy-compatible regularisation preserves useful decisions with modest utility loss. When stronger privacy is needed, the intrinsic guarantee can be sharpened by adding calibrated GP noise, providing an explicit additional privacy knob.

Problem

• Differential Privacy
• Gaussian Process
• Posterior Sampling
• Privacy Leakage
• Rényi-DP

Innovation

• Gaussian Process
• Differential Privacy
• Posterior Sampling
• Rényi-DP
• Regularization