A solvable model for unsupervised federated learning

2026-06-11

AI Summary

This work addresses the performance limitations of collaborative learning among participants (students) in unsupervised federated learning under heterogeneous data distributions. The authors propose a teacher–multi-student generative framework and, drawing upon equilibrium disordered systems theory and Bayesian inference, theoretically demonstrate for the first time that inter-student interactions can systematically enhance learning performance: students with high noise require fewer samples, while those with low noise achieve higher overlap with the true signal. They further derive the optimal Bayesian conditions for recovering the teacher’s signal and establish a formal mapping between the proposed framework and restricted Boltzmann machines. Numerical experiments quantitatively validate the influence of interaction strength, sample complexity, and noise levels on overall learning performance.

Abstract

We introduce a theoretical framework for analyzing federated learning in a generative setting through a teacher-multiple interacting students scenario, in which each student receives a distinct realization of the data, either through a different noise corruption or by accessing a different subset, possibly of varying size. Using theoretical tools in equilibrium disordered system, we analytically show that interactions among students systematically enhance learning performance: highly noisy students require fewer samples to recover the underlying pattern, while low-noise students achieve a larger overlap with the ground-truth signal. We derive the optimal Bayesian conditions for teacher recovery as functions of the sample complexity, noise level, and interaction strength, and validate these predictions through numerical simulations. The resulting dynamics can be mapped onto equilibrium sampling in a Restricted Boltzmann Machine with a structured hidden layer, providing a principled theoretical understanding of how interactions improve distributed generative modeling.

Problem

• unsupervised federated learning
• teacher-student model
• distributed generative modeling
• sample complexity
• noise corruption

Innovation

• unsupervised federated learning
• teacher-student model
• disordered systems
• Restricted Boltzmann Machine
• Bayesian inference