Federated Learning with Enhanced Privacy via Model Splitting and Random Client Participation

2025-09-30

AI Summary

In federated learning (FL), differential privacy (DP) noise severely degrades model accuracy, creating a fundamental tension between privacy and utility. To address this, we propose a novel framework integrating structured model partitioning with statistical privacy amplification: the model is decomposed into locally private submodules and globally shared submodules, with Gaussian noise injected only into the latter. We theoretically characterize the combined privacy amplification arising from both stochastic client participation and local data subsampling, enabling substantial reduction in the required noise magnitude to achieve a target $(varepsilon,delta)$-DP guarantee. Our approach thus preserves strict DP while significantly improving convergence speed and final model accuracy. Experiments on CIFAR-10 and CIFAR-100 demonstrate absolute accuracy gains of 3.2–7.8 percentage points over state-of-the-art DP-FL baselines, achieving a markedly superior privacy–utility trade-off.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Security and Privacy: Security and privacy of machine learning and AI applications
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Federated Learning (FL) often adopts differential privacy (DP) to protect client data, but the added noise required for privacy guarantees can substantially degrade model accuracy. To resolve this challenge, we propose model-splitting privacy-amplified federated learning (MS-PAFL), a novel framework that combines structural model splitting with statistical privacy amplification. In this framework, each client's model is partitioned into a private submodel, retained locally, and a public submodel, shared for global aggregation. The calibrated Gaussian noise is injected only into the public submodel, thereby confining its adverse impact while preserving the utility of the local model. We further present a rigorous theoretical analysis that characterizes the joint privacy amplification achieved through random client participation and local data subsampling under this architecture. The analysis provides tight bounds on both single-round and total privacy loss, demonstrating that MS-PAFL significantly reduces the noise necessary to satisfy a target privacy protection level. Extensive experiments validate our theoretical findings, showing that MS-PAFL consistently attains a superior privacy-utility trade-off and enables the training of highly accurate models under strong privacy guarantees.

Problem

• Enhancing privacy in federated learning while preserving model accuracy
• Reducing noise impact via model splitting and selective client participation
• Achieving superior privacy-utility trade-off under strong privacy guarantees

Innovation

• Model splitting into private and public submodels
• Injecting calibrated noise only into public submodels
• Amplifying privacy via random client participation