federated heterogeneity handling

In federated learning, the coexistence of statistical heterogeneity (non-IID data) and system heterogeneity (network latency) severely impedes convergence speed. Existing approaches address only one type of heterogeneity in isolation, lacking a unified optimization framework. This paper presents the first joint modeling of both heterogeneities and proposes two theoretically optimal client selection strategies per round. By minimizing the theoretical convergence time, we derive lightweight, efficiently solvable optimization problems that integrate latency-aware sampling, non-IID-adaptive training, and unified convergence analysis for both convex and non-convex objectives. Extensive experiments across nine non-IID datasets, two realistic latency distributions, and non-convex neural networks demonstrate that our method accelerates convergence by up to 20× over state-of-the-art baselines while ensuring stable convergence.

In federated learning, dual heterogeneity—across client communication capabilities and computational resources—induces misaligned optimization dynamics and objective inconsistency, causing the global model to converge to spurious stationary points deviating from the true optimum. This paper introduces the first unified theoretical framework analyzing communication-computation heterogeneity and proposes FedACS, a general client sampling scheme. FedACS employs a heterogeneity-aware dynamic sampling mechanism that rigorously eliminates all forms of objective inconsistency without modifying local solvers. We prove that FedACS achieves a convergence rate of $O(1/sqrt{R})$ under standard assumptions. Extensive experiments across diverse heterogeneous settings demonstrate that FedACS improves model accuracy by 4.3%–36%, reduces communication overhead by 22%–89%, and decreases per-round computational load by 14%–105%, significantly outperforming existing baselines.

To address poor generalization and slow convergence in federated learning (FL) caused by non-independent and identically distributed (Non-IID) client data, class/attribute imbalance, and spurious correlations, this paper proposes FedDiverse—a dynamic client selection algorithm. We further construct the first suite of seven vision benchmark datasets explicitly designed to capture multi-granularity imbalances and spurious correlations. Methodologically, we introduce, for the first time, a systematic six-dimensional metric for quantifying data heterogeneity and establish a novel client selection paradigm grounded in complementary distribution collaboration. Extensive experiments across the seven benchmarks demonstrate that FedDiverse consistently improves the average accuracy of mainstream FL methods by 3.2%, accelerates convergence by 21%, reduces communication and computational overhead, and enhances model robustness against distributional shifts and spurious patterns.

This work addresses the performance degradation of global models in federated learning caused by client data heterogeneity. To mitigate this issue, the authors propose Terraform, a novel approach that uniquely integrates gradient information with a deterministic client selection mechanism. By leveraging gradient updates to characterize client heterogeneity, Terraform enables precise participant selection, thereby enhancing both model accuracy and training efficiency. The method introduces a gradient-based heterogeneity metric coupled with a deterministic selection algorithm, yielding significant performance gains within the standard federated learning framework. Experimental results demonstrate that Terraform improves accuracy by up to 47% compared to existing methods, while ablation studies and training-time analyses further confirm its efficiency and robustness.

Federated learning faces concurrent heterogeneity across three dimensions: statistical (non-IID data distributions), data-quality (noisy or low-fidelity local datasets), and system (device-specific computation and communication delays), which existing methods struggle to jointly model and optimize. To address this, we propose a lightweight, flexible context-aware client selection framework. First, we unify the three heterogeneity types into a dynamic contextual representation and formulate client selection as a Contextual Multi-Armed Bandit (CMAB) problem, enabling heterogeneity-aware scoring and selection. Second, we introduce a lightweight coordination protocol to minimize system overhead. Our framework explicitly balances statistical utility and system efficiency. Extensive experiments demonstrate up to 10 percentage points absolute accuracy improvement over state-of-the-art methods across diverse heterogeneous settings. Moreover, it is fully compatible with and enhances mainstream robust aggregation algorithms—accelerating convergence and improving final model performance.

Federated learning faces significant challenges from data heterogeneity and partial client participation, leading to high update variance, deviation from optimal global convergence, degraded model performance, and slow training. To address these issues, this paper proposes a unified optimization framework integrating projection constraints and adaptive scaling. Specifically, local gradient updates are projected onto the direction of the previous global update to mitigate heterogeneity-induced bias, while an adaptive scaling mechanism is introduced to stabilize aggregation under partial participation. This work is the first to jointly model and suppress both variance sources in a cohesive manner. Extensive experiments on heterogeneous image classification benchmarks—including CIFAR-10, CIFAR-100, and Tiny-ImageNet—demonstrate substantial improvements in convergence speed and test accuracy, consistently outperforming state-of-the-art methods such as FedAvg, FedProx, and SCAFFOLD.

This paper addresses the “heterogeneity amplification” problem in asynchronous federated learning (AFL), wherein uneven client participation—especially frequent updates from fast clients under non-IID data—exacerbates global model bias and slows convergence. To mitigate this, we propose a full-client participation mechanism and design two buffer-free, immediate-update algorithms: ACE (Asynchronous Client Engagement) and its delay-aware variant ACED. Both employ dynamic participation control, latency-adaptive weight adjustment, and full-client gradient aggregation, resolving the tension between update staleness and client heterogeneity without introducing auxiliary storage overhead. Theoretical analysis establishes convergence guarantees under realistic asynchrony and heterogeneity. Extensive experiments across multiple tasks demonstrate that ACE and ACED significantly improve convergence speed and model robustness—particularly in high-heterogeneity and high-latency regimes—outperforming state-of-the-art AFL baselines.

To address the client drift and generalization imbalance in federated learning under non-independent and identically distributed (Non-IID) data, this paper identifies a critical limitation of existing personalization methods: their excessive focus on local accuracy while neglecting out-of-distribution (OOD) generalization—a fundamental pillar of FedAvg’s robustness. We propose a unified evaluation paradigm that jointly optimizes local accuracy and OOD generalization, and design FLIU, an adaptive personalization update mechanism. Within the FedAvg framework, FLIU introduces learnable, client-specific scaling factors to dynamically balance global consistency and local adaptability. Extensive experiments across MNIST and CIFAR-10 under IID, pathological Non-IID, and Dirichlet Non-IID settings demonstrate that FLIU achieves high local accuracy while significantly improving OOD generalization—outperforming state-of-the-art personalized federated learning methods.

本文针对联邦学习中学习者与客户端分布不匹配问题，提出了一种基于代理数据集动态评估并选择对优化目标贡献大的客户端的方法。

This work addresses the challenge in federated learning where client data heterogeneity and anomalous behaviors often lead to unstable model updates, complicating the distinction between benign distribution shifts and harmful outliers. To this end, the paper proposes a lightweight, permutation-invariant geometric divergence metric that leverages a shared probing set to analyze discrepancies in how local and global models partition the input space at the representation level. By focusing on functional behavior in the representation space rather than model parameters or gradients, the method accurately quantifies each client’s functional deviation and effectively discriminates between stably heterogeneous clients and truly anomalous ones, thereby providing a reliable basis for risk-aware aggregation.