Bounding Reconstruction Attack Success of Adversaries Without Data Priors

2024-02-20

arXiv.org

AI Summary

In differential privacy (DP), the privacy budget ε lacks a quantitative mapping to the risk of training data reconstruction attacks, especially under realistic assumptions—i.e., without data priors and beyond worst-case scenarios. Method: We propose the first verifiable, formal upper bound on training data reconstruction success rate, integrating rigorous DP theoretical analysis, information-theoretic bounding techniques, and gradient-leakage modeling. The bound is empirically validated via inversion attacks and other reconstruction benchmarks. Contribution/Results: Our bound is tight, supports multi-metric and multi-scenario ε selection, and—crucially—establishes the first quantitative relationship between ε and reconstruction success probability. It enables principled, scientifically grounded DP hyperparameter tuning, significantly improving the efficiency and rigor of the privacy–utility trade-off.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Adversarial Attacks & Robustness
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Reconstruction attacks on machine learning (ML) models pose a strong risk of leakage of sensitive data. In specific contexts, an adversary can (almost) perfectly reconstruct training data samples from a trained model using the model's gradients. When training ML models with differential privacy (DP), formal upper bounds on the success of such reconstruction attacks can be provided. So far, these bounds have been formulated under worst-case assumptions that might not hold high realistic practicality. In this work, we provide formal upper bounds on reconstruction success under realistic adversarial settings against ML models trained with DP and support these bounds with empirical results. With this, we show that in realistic scenarios, (a) the expected reconstruction success can be bounded appropriately in different contexts and by different metrics, which (b) allows for a more educated choice of a privacy parameter.

Problem

• Deriving formal privacy bounds for data reconstruction attacks
• Bridging the gap between theoretical DP guarantees and real-world threats
• Establishing probabilistic success bounds for from-scratch gradient inversion attacks

Innovation

• Derives formal privacy bounds for gradient inversion attacks
• Models from-scratch attacks as mean estimation problem
• Provides probabilistic success bounds using MSE and PSNR