🤖 AI Summary
This work addresses the challenges of high communication overhead and privacy leakage in non-IID federated learning by proposing a novel approach that integrates differential privacy with adaptive quantization. The method employs the Laplace mechanism to provide rigorous privacy guarantees and introduces a global bit-width scheduler based on cosine annealing across communication rounds, coupled with a client-aware quantization strategy informed by dataset entropy, to dynamically optimize the volume of uploaded data. Experimental results demonstrate significant communication reductions—up to 52.64% on MNIST, 45.06% on CIFAR-10, and 31%–37% on medical imaging datasets—while preserving model accuracy, thereby achieving a synergistic optimization of communication efficiency and strong privacy protection.
📝 Abstract
Federated learning (FL) is a distributed machine learning method where multiple devices collaboratively train a model under the management of a central server without sharing underlying data. One of the key challenges of FL is the communication bottleneck caused by variations in connection speed and bandwidth across devices. Therefore, it is essential to reduce the size of transmitted data during training. Additionally, there is a potential risk of exposing sensitive information through the model or gradient analysis during training. To address both privacy and communication efficiency, we combine differential privacy (DP) and adaptive quantization methods. We use Laplacian-based DP to preserve privacy, which is relatively underexplored in FL and offers tighter privacy guarantees than Gaussian-based DP. We propose a simple and efficient global bit-length scheduler using round-based cosine annealing, along with a client-based scheduler that dynamically adapts based on client contribution estimated through dataset entropy analysis. We evaluate our approach through extensive experiments on CIFAR10, MNIST, and medical imaging datasets, using non-IID data distributions across varying client counts, bit-length schedulers, and privacy budgets. The results show that our adaptive quantization methods reduce total communicated data by up to 52.64% for MNIST, 45.06% for CIFAR10, and 31% to 37% for medical imaging datasets compared to 32-bit float training while maintaining competitive model accuracy and ensuring robust privacy through differential privacy.