Certification-Based Differentially Private Learning

2026-09-30

AI Summary

This study addresses the privacy-utility imbalance caused by global sensitivity in differential privacy and the absence of finite privacy guarantees for unbounded regression. It pioneers the extension of abstract interpretation to private prediction for continuous unbounded regression. By proposing an Abstract Gradient Sampling (AGS) algorithm alongside a smooth sensitivity upper-bounding technique, this work reformulates parameter learning as a regression problem, enabling formal certification of private learning via reachability analysis. Experimental results demonstrate that the derived regression bounds are tighter than those obtained using global sensitivity baselines. Furthermore, the proposed approach achieves the first finite privacy guarantees in unbounded settings and yields private learning performance superior to standard algorithms under matched conditions.

Abstract

Differential privacy (DP) in machine learning is typically achieved by adding noise to model parameters (private learning) or to model outputs (private prediction). Recent work uses formal methods, namely abstract interpretation, to provide tighter privacy guarantees, but only for private prediction in classification settings. In this work, we investigate the use of formal methods as a general tool for tighter privacy analysis. First, we generalize the abstract gradient training (AGT) framework to private prediction in continuous, unbounded regression. Second, by reducing learning in parameterized models to a regression problem over the parameter space, we introduce Abstract Gradient Sampling (AGS), an algorithm that enables reachability-based analysis to provide guarantees for private learning. In both private prediction and private learning, we provide tightened privacy accounting for the AGT framework and a theoretical analysis demonstrating when our smooth sensitivity upper-bounds yield favourable privacy-utility trade-off. In practice, we validate that our regression bounds are tighter than global-sensitivity baselines on regression benchmarks, and, notably, yield the first finite privacy guarantees in settings where global prediction sensitivity is a priori unbounded. We also find that under matched conditions, our private learning algorithm can outperform standard private learners.

Problem

• Differential Privacy
• Private Learning
• Private Prediction
• Formal Methods
• Regression

Innovation

• Differential Privacy
• Abstract Interpretation
• Abstract Gradient Sampling
• Smooth Sensitivity
• Private Learning