A Federated Many-to-One Hopfield model for associative Neural Networks

2026-03-20

AI Summary

This work addresses the challenges of convergence difficulty and catastrophic forgetting in federated learning under client heterogeneity and data distribution drift. The authors propose a federated associative memory framework that enables continual learning while preserving privacy by constructing low-rank Hebbian operators locally at clients and aggregating them at the server to extract global prototypes. A novel federated aggregation mechanism based on low-rank spectral inference is introduced, coupled with an entropy controller that dynamically balances model stability and plasticity, thereby accommodating data evolution and noise without requiring a central replay buffer. Experimental results demonstrate that the proposed method significantly improves global prototype reconstruction quality and associative retrieval performance in heterogeneous, drifting, and novel-class scenarios, validating the efficacy of spectral-based federated consolidation.

Technology Category

Machine Learning: Distributed Machine Learning & Federated LearningMultiagent Systems: Multiagent LearningSearch and Optimization: Learning to Search

Application Category

User Modeling, Personalization and Recommendation: Federated recommendation systems and personalizationSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingGraph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Federated learning enables collaborative training without sharing raw data, but struggles under client heterogeneity and streaming distribution shifts, where drift and novel data can impair convergence and cause forgetting. We propose a federated associative-memory framework that learns shared archetypes in heterogeneous, continual settings, where client data are independent but not necessarily balanced. Each client encodes its experience as a low-rank Hebbian operator, sent to a central server for aggregation and factorization into global archetypes. This approach preserves privacy, avoids centralized replay buffers, and is robust to small, noisy, or evolving datasets. We cast aggregation as a low-rank-plus-noise spectral inference problem, deriving theoretical thresholds for detectability and retrieval robustness. An entropy-based controller balances stability and plasticity in streaming regimes. Experiments with heterogeneous clients, drift, and novelty show improved global archetype reconstruction and associative retrieval, supporting the spectral view of federated consolidation.

Problem

• federated learning
• client heterogeneity
• distribution shift
• catastrophic forgetting
• continual learning

Innovation

• Federated Learning
• Associative Memory
• Low-rank Hebbian Operators
• Spectral Inference
• Continual Learning