Robust Federated Learning Over the Air: Combating Heavy-Tailed Noise with Median Anchored Clipping

2024-09-23

arXiv.org

AI Summary

In over-the-air computation (AirComp)-enabled analog federated learning, heavy-tailed electromagnetic interference severely distorts global gradients, degrading training convergence and model accuracy. Method: This paper proposes Median-anchored Clipping (MAC), the first gradient clipping scheme that dynamically anchors the clipping threshold to the sample median—ensuring robustness against heavy-tailed noise while maintaining low computational overhead. We rigorously derive a closed-form convergence rate for AirComp-based federated learning with MAC and jointly optimize gradient clipping and channel characteristics through heavy-tailed noise modeling and analysis. Results: Experiments across diverse wireless channels demonstrate that MAC significantly mitigates heavy-tailed interference, enhancing convergence stability and final model accuracy. The method achieves communication-computation co-design for robust optimization, outperforming conventional clipping strategies in both theoretical guarantees and empirical performance.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Multiagent Systems: Agent Communication
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Leveraging over-the-air computations for model aggregation is an effective approach to cope with the communication bottleneck in federated edge learning. By exploiting the superposition properties of multi-access channels, this approach facilitates an integrated design of communication and computation, thereby enhancing system privacy while reducing implementation costs. However, the inherent electromagnetic interference in radio channels often exhibits heavy-tailed distributions, giving rise to exceptionally strong noise in globally aggregated gradients that can significantly deteriorate the training performance. To address this issue, we propose a novel gradient clipping method, termed Median Anchored Clipping (MAC), to combat the detrimental effects of heavy-tailed noise. We also derive analytical expressions for the convergence rate of model training with analog over-the-air federated learning under MAC, which quantitatively demonstrates the effect of MAC on training performance. Extensive experimental results show that the proposed MAC algorithm effectively mitigates the impact of heavy-tailed noise, hence substantially enhancing system robustness.

Problem

• Airborne Computing
• Radio Interference
• Model Training Degradation

Innovation

• Median Anchor Clipping (MAC)
• Radio Interference Mitigation
• Enhanced Model Learning Stability