🤖 AI Summary
To address the challenge of balancing privacy preservation and model utility in pre-trained models, this paper proposes DP-Weights—the first post-training differential privacy (DP) method. DP-Weights injects theoretically calibrated noise directly into model weights, enabling flexible privacy–utility trade-off control without retraining and providing rigorous, formally verified DP guarantees. On CIFAR-10, DP-Weights achieves only a 0.8% accuracy drop at ε ≈ 2.5, matching the statistical utility of DP-SGD while reducing training overhead by over 90%. Unlike DP-SGD—which requires noise injection during training—DP-Weights breaks this paradigm constraint, offering an efficient, plug-and-play solution for privacy-enhancing model deployment. This work establishes a new, practical pathway for retrofitting privacy into existing models without sacrificing performance or incurring prohibitive computational costs.
📝 Abstract
Imagine training a machine learning model with Differentially Private Stochastic Gradient Descent (DP-SGD), only to discover post-training that the noise level was either too high, crippling your model's utility, or too low, compromising privacy. The dreaded realization hits: you must start the lengthy training process from scratch. But what if you could avoid this retraining nightmare? In this study, we introduce a groundbreaking approach (to our knowledge) that applies differential privacy noise to the model's weights after training. We offer a comprehensive mathematical proof for this novel approach's privacy bounds, use formal methods to validate its privacy guarantees, and empirically evaluate its effectiveness using membership inference attacks and performance evaluations. This method allows for a single training run, followed by post-hoc noise adjustments to achieve optimal privacy-utility trade-offs. We compare this novel fine-tuned model (DP-Weights model) to a traditional DP-SGD model, demonstrating that our approach yields statistically similar performance and privacy guarantees. Our results validate the efficacy of post-training noise application, promising significant time savings and flexibility in fine-tuning differential privacy parameters, making it a practical alternative for deploying differentially private models in real-world scenarios.