Combining Homomorphic Encryption and Differential Privacy in Federated Learning for Model Inspection and Availability

2026-10-01

AI Summary

This study addresses the challenge in federated learning where single privacy-preserving mechanisms struggle to simultaneously maintain model utility and ensure data security. To overcome this limitation, we propose a novel synergistic framework that integrates homomorphic encryption (HE) for training with differential privacy (DP) for auditing. Methodologically, Markov Chain Monte Carlo (MCMC) Bayesian inference is employed to precisely estimate privacy parameters, thereby transcending the constraints of conventional single-mechanism approaches. Experimental evaluations on the FEMNIST dataset demonstrate that the proposed framework reduces the test loss to 1.09 while optimizing the privacy budget ε to 4.32. These results significantly outperform pure DP baselines, achieving an effective balance between high model utility and robust privacy guarantees.

Abstract

The increasing prevalence of decentralized data has led to a growing interest in federated learning, which enables collaborative model training without clients sharing their sensitive local data. However, FL alone does not sufficiently protect sensitive training data and is generally coupled with privacy-preserving techniques, such as differential privacy and homomorphic encryption. Although powerful, these techniques address separate concerns via different mechanisms, so relying on just one might prove insufficient or impractical for addressing challenges associated with federated learning. In this work, we propose a privacy-preserving federated learning framework that combines homomorphic encryption-based training with differential privacy-based model inspection and release. We adopt a Markov chain Monte Carlo-based Bayesian privacy estimation method to estimate the privacy of our proposed framework. Our results show that this method improves both model utility and estimated privacy over the baseline method that relies solely on differential privacy for training. In our experiments with the FEMNIST dataset, by the end of training, our method reaches a test loss of $1.09$, compared to $2.37$ for the differential privacy-only approach, while providing stronger estimated privacy protection, with the estimated posterior mean of the privacy parameter $ε$ of $4.32$, compared to $7.26$ for the differential privacy-only approach. We also show that intermittent model monitoring can preserve the encrypted training trajectory while, under our evaluated experimental setting, providing estimated privacy comparable to or stronger than the differential privacy-only approach.

Problem

• Federated Learning
• Homomorphic Encryption
• Differential Privacy
• Privacy Preservation
• Model Utility

Innovation

• Federated Learning
• Homomorphic Encryption
• Differential Privacy
• Bayesian Privacy Estimation
• Markov Chain Monte Carlo