SAFL: Structure-Aware Personalized Federated Learning via Client-Specific Clustering and SCSI-Guided Model Pruning

2025-01-30

AI Summary

In heterogeneous federated learning, personalized model training faces challenges of high computational and communication overheads, as well as pruning mismatch caused by scarce local data. To address these issues, this paper proposes a lightweight framework integrating client clustering with similarity-driven structured channel-wise pruning (SCSI). Specifically, it employs similarity-guided client clustering, a joint pruning criterion based on SCSI, and a two-stage prune-train-aggregate paradigm to enable structural-aware customization of personalized submodels. The method significantly reduces model size and communication load while preserving model accuracy and improving inference performance. Experimental results demonstrate superior performance over state-of-the-art personalized federated learning approaches across multiple heterogeneous benchmarks. Notably, this work achieves, for the first time, deep integration of clustering-guided personalization and structured pruning within the federated learning paradigm.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Security and Privacy: Privacy-enhancing technologies

Abstract

Federated Learning (FL) enables clients to collaboratively train machine learning models without sharing local data, preserving privacy in diverse environments. While traditional FL approaches preserve privacy, they often struggle with high computational and communication overhead. To address these issues, model pruning is introduced as a strategy to streamline computations. However, existing pruning methods, when applied solely based on local data, often produce sub-models that inadequately reflect clients' specific tasks due to data insufficiency. To overcome these challenges, this paper introduces SAFL (Structure-Aware Federated Learning), a novel framework that enhances personalized federated learning through client-specific clustering and Similar Client Structure Information (SCSI)-guided model pruning. SAFL employs a two-stage process: initially, it groups clients based on data similarities and uses aggregated pruning criteria to guide the pruning process, facilitating the identification of optimal sub-models. Subsequently, clients train these pruned models and engage in server-based aggregation, ensuring tailored and efficient models for each client. This method significantly reduces computational overhead while improving inference accuracy. Extensive experiments demonstrate that SAFL markedly diminishes model size and improves performance, making it highly effective in federated environments characterized by heterogeneous data.

Problem

• Privacy-preserving Federated Learning
• Model Training Efficiency
• Data Transmission Optimization

Innovation

• SAFL
• Federated Learning
• Data Similarity