🤖 AI Summary
This work addresses the performance degradation of federated learning under data heterogeneity—including label shift, covariate shift, and concept shift—as well as in data-scarce settings. The authors propose a communication-free personalization method that dynamically conditions a single global model by embedding client-specific PCA statistics, computed locally, as continuous conditional inputs. Trained within the standard federated learning framework, the approach incurs no additional communication overhead. Extensive experiments demonstrate that the method consistently outperforms existing approaches across 97 configurations, surpassing even an oracle baseline with access to true cluster assignments by 1–6% under complex heterogeneity, while maintaining robust performance under data sparsity—where it remains the only method exhibiting consistent stability.
📝 Abstract
Federated learning (FL) under data heterogeneity remains challenging: existing methods either ignore client differences (FedAvg), require costly cluster discovery (IFCA), or maintain per-client models (Ditto). All degrade when data is sparse or heterogeneity is multi-dimensional. We propose conditioning a single global model on locally-computed PCA statistics of each client's training data, requiring zero additional communication. Evaluating across 97~configurations spanning four heterogeneity types (label shift, covariate shift, concept shift, and combined heterogeneity), four datasets (MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100), and seven FL baseline methods, we find that our method matches the Oracle baseline -- which knows true cluster assignments -- across all settings, surpasses it by 1--6% on combined heterogeneity where continuous statistics are richer than discrete cluster identifiers, and is uniquely sparsity-robust among all tested methods.