Latent Information Sharing for Accelerating Federated Learning

2026-10-01

AI Summary

This study addresses the challenge of client drift in federated learning caused by data heterogeneity, which severely hinders global model convergence and training efficiency. To mitigate this issue, we propose a distributed optimization framework based on latent information sharing that aggregates cross-client knowledge by exchanging a minimal set of hidden-layer activations, thereby alleviating data heterogeneity while preserving privacy guarantees. We provide theoretical convergence analysis for the proposed method and demonstrate through extensive experiments that it significantly outperforms baseline approaches such as FedProx under a fixed communication budget. Notably, the framework substantially improves model accuracy without introducing additional communication overhead, validating the effectiveness of activation sharing mechanisms for federated optimization.

Abstract

Federated learning (FL) is a communication-efficient distributed learning paradigm. However, client drift remains one of the most critical challenges, hindering the efficient training of a global model. In this study, we propose a novel latent information sharing scheme that directly mitigates data heterogeneity across clients. Our theoretical and empirical results show that sharing a small amount of hidden-layer activations significantly improves training efficiency while preserving convergence guarantees and data privacy. Furthermore, we compare our method with existing FL approaches designed to address client drift, including FedProx, SCAFFOLD, FedPVR, FedProto, and SplitFed, and demonstrate superior model accuracy under a fixed round budget without incurring excessive communication overhead. Overall, this work presents a promising new knowledge aggregation scheme and provides a comprehensive analysis of the impact of activation sharing on federated optimization.

Problem

• Federated Learning
• Client Drift
• Data Heterogeneity
• Distributed Learning

Innovation

• Federated Learning
• Client Drift
• Latent Information Sharing
• Activation Sharing
• Knowledge Aggregation