Decoupled Contrastive Learning for Federated Learning

2025-08-05

AI Summary

In federated learning (FL), client data heterogeneity degrades model performance, while contrastive learning suffers from insufficient negative samples in few-shot FL settings—violating the “infinite negatives” assumption. To address these issues, we propose Decoupled Contrastive Federated Learning (DCFL). DCFL is the first framework to theoretically characterize the few-shot limitations of contrastive learning under federated settings. It innovatively decouples the contrastive loss into two independent objectives: alignment (encouraging similarity between positive pairs) and uniformity (promoting even distribution of representations on the hypersphere), thereby eliminating reliance on large-scale negative sampling. Clients jointly optimize both objectives locally, enhancing representation generalizability. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that DCFL significantly improves classification accuracy and robustness, while simultaneously strengthening positive-pair alignment and negative-sample distribution uniformity.

Technology Category

Machine Learning: Distributed Machine Learning & Federated LearningComputer Vision: Learning & Optimization for CVSearch and Optimization: Learning to Search

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingUser Modeling, Personalization and Recommendation: Federated recommendation systems and personalizationWeb Mining and Content Analysis: Large pretrained models with web data

Abstract

Federated learning is a distributed machine learning paradigm that allows multiple participants to train a shared model by exchanging model updates instead of their raw data. However, its performance is degraded compared to centralized approaches due to data heterogeneity across clients. While contrastive learning has emerged as a promising approach to mitigate this, our theoretical analysis reveals a fundamental conflict: its asymptotic assumptions of an infinite number of negative samples are violated in finite-sample regime of federated learning. To address this issue, we introduce Decoupled Contrastive Learning for Federated Learning (DCFL), a novel framework that decouples the existing contrastive loss into two objectives. Decoupling the loss into its alignment and uniformity components enables the independent calibration of the attraction and repulsion forces without relying on the asymptotic assumptions. This strategy provides a contrastive learning method suitable for federated learning environments where each client has a small amount of data. Our experimental results show that DCFL achieves stronger alignment between positive samples and greater uniformity between negative samples compared to existing contrastive learning methods. Furthermore, experimental results on standard benchmarks, including CIFAR-10, CIFAR-100, and Tiny-ImageNet, demonstrate that DCFL consistently outperforms state-of-the-art federated learning methods.

Problem

• Addresses performance degradation in federated learning due to data heterogeneity
• Resolves conflict between contrastive learning assumptions and finite-sample federated learning
• Improves alignment and uniformity in federated contrastive learning with limited client data

Innovation

• Decouples contrastive loss into two objectives
• Independent calibration of attraction and repulsion forces
• Suitable for federated learning with small client data