SVDefense: Effective Defense against Gradient Inversion Attacks via Singular Value Decomposition

2025-09-30

AI Summary

Gradient inversion attacks (GIA) in federated learning enable adversaries to reconstruct private training data from shared gradients. Existing defenses struggle to jointly ensure privacy preservation, model utility, and efficiency for embedded deployment, and are often vulnerable to adaptive attacks. This paper proposes a lightweight defense framework based on truncated singular value decomposition (SVD), featuring three key innovations: (1) an adaptive energy threshold for gradient truncation, (2) channel-weighted gradient approximation, and (3) a hierarchical weighted aggregation mechanism—collectively enabling implicit obfuscation of sensitive information during gradient compression. The method requires no additional noise injection or cryptographic overhead, significantly reducing computational cost and supporting resource-constrained devices. Evaluated on image classification, human activity recognition, and keyword spotting tasks, it achieves an average 2.1–4.7% higher model accuracy than state-of-the-art defenses under equivalent privacy guarantees, while effectively resisting white-box adaptive GIA.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Large-scale security measurements

Abstract

Federated learning (FL) enables collaborative model training without sharing raw data but is vulnerable to gradient inversion attacks (GIAs), where adversaries reconstruct private data from shared gradients. Existing defenses either incur impractical computational overhead for embedded platforms or fail to achieve privacy protection and good model utility at the same time. Moreover, many defenses can be easily bypassed by adaptive adversaries who have obtained the defense details. To address these limitations, we propose SVDefense, a novel defense framework against GIAs that leverages the truncated Singular Value Decomposition (SVD) to obfuscate gradient updates. SVDefense introduces three key innovations, a Self-Adaptive Energy Threshold that adapts to client vulnerability, a Channel-Wise Weighted Approximation that selectively preserves essential gradient information for effective model training while enhancing privacy protection, and a Layer-Wise Weighted Aggregation for effective model aggregation under class imbalance. Our extensive evaluation shows that SVDefense outperforms existing defenses across multiple applications, including image classification, human activity recognition, and keyword spotting, by offering robust privacy protection with minimal impact on model accuracy. Furthermore, SVDefense is practical for deployment on various resource-constrained embedded platforms. We will make our code publicly available upon paper acceptance.

Problem

• Defend against gradient inversion attacks in federated learning
• Balance privacy protection with model utility effectively
• Provide practical defense for resource-constrained embedded platforms

Innovation

• Uses truncated SVD for gradient obfuscation
• Self-adaptive energy threshold for client vulnerability
• Channel-wise weighted approximation for selective preservation