Redefining Clustered Federated Learning for System Identification: The Path of ClusterCraft

2025-05-22

AI Summary

Addressing the challenges of prior-knowledge scarcity, data heterogeneity-induced clustering instability, and model redundancy in multi-source system identification (SYSID), this paper proposes IC-SYSID—an incremental clustering federated learning framework. Methodologically, it introduces (1) ClusterCraft, the first incremental clustering algorithm for dynamic and scalable client grouping; and (2) ClusterMerge, a model fusion mechanism integrating scaled Glorot initialization with an ℓ₂-regularized loss term to enhance collaborative modeling stability and generalization. Evaluated on a real-world fleet vehicle dynamics joint identification task, IC-SYSID reduces the number of clusters by 37% and lowers average identification error by 21.4% compared to baselines, while effectively suppressing unstable cluster formation. The framework delivers an efficient and robust federated solution for prior-free, multi-source SYSID.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Multiagent Systems: Multiagent Learning
• Intelligent Robots: State Estimation

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

This paper addresses the System Identification (SYSID) problem within the framework of federated learning. We introduce a novel algorithm, Incremental Clustering-based federated learning method for SYSID (IC-SYSID), designed to tackle SYSID challenges across multiple data sources without prior knowledge. IC-SYSID utilizes an incremental clustering method, ClusterCraft (CC), to eliminate the dependency on the prior knowledge of the dataset. CC starts with a single cluster model and assigns similar local workers to the same clusters by dynamically increasing the number of clusters. To reduce the number of clusters generated by CC, we introduce ClusterMerge, where similar cluster models are merged. We also introduce enhanced ClusterCraft to reduce the generation of similar cluster models during the training. Moreover, IC-SYSID addresses cluster model instability by integrating a regularization term into the loss function and initializing cluster models with scaled Glorot initialization. It also utilizes a mini-batch deep learning approach to manage large SYSID datasets during local training. Through the experiments conducted on a real-world representing SYSID problem, where a fleet of vehicles collaboratively learns vehicle dynamics, we show that IC-SYSID achieves a high SYSID performance while preventing the learning of unstable clusters.

Problem

• Solving System Identification in federated learning without prior knowledge
• Reducing cluster dependency via incremental clustering and merging
• Enhancing cluster stability with regularization and optimized initialization

Innovation

• Incremental clustering method for federated learning
• ClusterMerge to reduce redundant clusters
• Regularization and Glorot initialization for stability