Federated Gaussian Mixture Models

2025-06-02

AI Summary

To address the challenges of statistical heterogeneity, high communication overhead, and privacy constraints in federated Gaussian Mixture Model (GMM) learning, this paper proposes FedGenGMM—the first single-round-aggregation federated GMM framework. FedGenGMM avoids raw data sharing and accommodates heterogeneous model complexities across clients. Leveraging the generative nature of GMMs, it performs local training followed by only one server-side aggregation of sufficient statistics. Innovatively integrating synthetic data generation with lightweight parameter aggregation, FedGenGMM simultaneously ensures strong privacy preservation, ultra-low communication cost (reducing bandwidth by over 90%), and robust generalization. Extensive experiments on anomaly detection across image, tabular, and time-series datasets demonstrate that FedGenGMM matches the performance of centralized training and multi-round federated baselines, while significantly improving computational efficiency and robustness to client heterogeneity.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Natural Language Processing: Generation
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

This paper introduces FedGenGMM, a novel one-shot federated learning approach for Gaussian Mixture Models (GMM) tailored for unsupervised learning scenarios. In federated learning (FL), where multiple decentralized clients collaboratively train models without sharing raw data, significant challenges include statistical heterogeneity, high communication costs, and privacy concerns. FedGenGMM addresses these issues by allowing local GMM models, trained independently on client devices, to be aggregated through a single communication round. This approach leverages the generative property of GMMs, enabling the creation of a synthetic dataset on the server side to train a global model efficiently. Evaluation across diverse datasets covering image, tabular, and time series data demonstrates that FedGenGMM consistently achieves performance comparable to non-federated and iterative federated methods, even under significant data heterogeneity. Additionally, FedGenGMM significantly reduces communication overhead, maintains robust performance in anomaly detection tasks, and offers flexibility in local model complexities, making it particularly suitable for edge computing environments.

Problem

• One-shot federated learning for Gaussian Mixture Models
• Addressing statistical heterogeneity and communication costs in FL
• Enhancing privacy and efficiency in unsupervised edge computing

Innovation

• One-shot federated learning for GMM
• Synthetic dataset for global model training
• Reduced communication costs in FL