High-Energy Concentration for Federated Learning in Frequency Domain

2025-09-15

AI Summary

To address the high communication overhead and redundant noise induced by data heterogeneity in federated learning (FL), this paper proposes FedFD, a frequency-domain-driven efficient FL framework. Its core innovation lies in the first integration of frequency-domain analysis into federated data distillation: discrete cosine transform (DCT) is employed to concentrate on high-energy low-frequency components; a binary mask filters out high-frequency noise; and a frequency-domain distribution alignment mechanism is introduced to improve global model convergence. Furthermore, a real-data-guided synthetic classification loss enhances low-frequency representation quality. FedFD achieves joint optimization of communication compression and model performance while preserving privacy. Extensive experiments across five image and speech datasets—including CIFAR-10 under extreme non-IID settings (Dirichlet α = 0.01)—demonstrate that FedFD reduces communication cost by up to 37.78% and improves accuracy by up to 10.88% over state-of-the-art methods.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Diffusion Models for Vision
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Federated Learning (FL) presents significant potential for collaborative optimization without data sharing. Since synthetic data is sent to the server, leveraging the popular concept of dataset distillation, this FL framework protects real data privacy while alleviating data heterogeneity. However, such methods are still challenged by the redundant information and noise in entire spatial-domain designs, which inevitably increases the communication burden. In this paper, we propose a novel Frequency-Domain aware FL method with high-energy concentration (FedFD) to address this problem. Our FedFD is inspired by the discovery that the discrete cosine transform predominantly distributes energy to specific regions, referred to as high-energy concentration. The principle behind FedFD is that low-energy like high-frequency components usually contain redundant information and noise, thus filtering them helps reduce communication costs and optimize performance. Our FedFD is mathematically formulated to preserve the low-frequency components using a binary mask, facilitating an optimal solution through frequency-domain distribution alignment. In particular, real data-driven synthetic classification is imposed into the loss to enhance the quality of the low-frequency components. On five image and speech datasets, FedFD achieves superior performance than state-of-the-art methods while reducing communication costs. For example, on the CIFAR-10 dataset with Dirichlet coefficient $α= 0.01$, FedFD achieves a minimum reduction of 37.78% in the communication cost, while attaining a 10.88% performance gain.

Problem

• Reducing communication burden in federated learning
• Filtering redundant information and noise
• Preserving privacy while optimizing performance

Innovation

• Frequency-domain filtering with binary mask
• Energy concentration for communication reduction
• Loss function enhances low-frequency components