DROP: Poison Dilution via Knowledge Distillation for Federated Learning

2025-02-10

AI Summary

Federated learning faces severe threats from stealthy backdoor attacks, particularly under non-IID data distributions with low poisoning rates and multiple malicious clients—scenarios where existing defenses struggle to reliably detect and mitigate such attacks. To address this, we propose a robust defense framework that introduces knowledge distillation into federated learning for the first time, synergistically integrating client clustering, update activity tracking, and cross-client knowledge distillation to dynamically adapt to heterogeneous data distributions and varying fractions of malicious participants. Our method significantly enhances detection accuracy and generalization robustness against stealthy backdoor attacks: across diverse non-IID settings, the backdoor success rate drops below 3%, while the global model maintains an accuracy above 92%. Extensive experiments demonstrate consistent superiority over state-of-the-art defenses in both security and utility.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Federated Learning is vulnerable to adversarial manipulation, where malicious clients can inject poisoned updates to influence the global model's behavior. While existing defense mechanisms have made notable progress, they fail to protect against adversaries that aim to induce targeted backdoors under different learning and attack configurations. To address this limitation, we introduce DROP (Distillation-based Reduction Of Poisoning), a novel defense mechanism that combines clustering and activity-tracking techniques with extraction of benign behavior from clients via knowledge distillation to tackle stealthy adversaries that manipulate low data poisoning rates and diverse malicious client ratios within the federation. Through extensive experimentation, our approach demonstrates superior robustness compared to existing defenses across a wide range of learning configurations. Finally, we evaluate existing defenses and our method under the challenging setting of non-IID client data distribution and highlight the challenges of designing a resilient FL defense in this setting.

Problem

• Federated Learning vulnerability
• Targeted backdoor attacks
• Non-IID data distribution

Innovation

• Knowledge distillation reduces poisoning
• Clustering and activity-tracking enhance security
• Effective in non-IID data distributions