Multi-Level Analyzation of Imbalance to Resolve Non-IID-Ness in Federated Learning

2026-06-08

AI Summary

This work addresses the performance degradation in federated learning caused by class imbalance-induced non-IID data distributions. It systematically formulates a three-tiered imbalance framework encompassing inter-sample, inter-class, and inter-client heterogeneity. To tackle this challenge, the authors propose FedBB, a lightweight approach that integrates a positive-negative balancing (PNB) loss during local training and a client-balanced reweighting (CBR) strategy during global aggregation, both relying solely on minimal statistical information to preserve privacy. FedBB demonstrates strong generalizability and practicality, serving as a versatile baseline for both generic and personalized federated learning. Extensive experiments on X-ray and natural image datasets show that FedBB significantly outperforms existing methods in terms of accuracy and efficiency, with ablation studies confirming the effectiveness of its individual components.

Abstract

Class imbalance is a common problem in deep learning that severely degrades performance. In federated learning (FL), it is a critical factor contributing to non-identically distributed data (non-IID). Building on several previous attempts, we define and analyze imbalance issues in FL at three levels: inter-case, inter-class, and inter-client. Inter-case imbalance addresses the imbalance in every single class; inter-class imbalance compares the number of data between different classes. Inter-client imbalance represents different skewness of local data between clients. Based on these concepts, we propose FedBB, which consists of two main components: (1) Positive Negative Balanced (PNB) loss function addresses the inter-case and inter-class imbalances in local training, enhancing generalization on highly skewed local client datasets. It optimizes both multi-label and multi-class classifications by assigning higher weights to minority cases or classes. (2) Client Balanced Reweighting (CBR) reweights clients based on inter-client imbalance during model aggregation, giving greater weight to models trained on less skewed datasets. Various experiments on X-ray and natural image datasets demonstrate that FedBB outperforms other algorithms in both performance and efficiency. Additionally, it requires limited statistical information, which is beneficial for privacy protection. Through ablation studies, we proved that PNB loss and CBR independently contribute to performance. As FedBB aims to build a global model that accurately classifies all classes, it can serve as a baseline for the generic and personalized FL.

Problem

• class imbalance
• non-IID
• federated learning
• inter-client imbalance
• inter-class imbalance

Innovation

• Federated Learning
• Non-IID
• Class Imbalance
• Loss Reweighting
• Client Aggregation