Federated Learning with Feedback Alignment

2025-12-14

AI Summary

Federated learning (FL) suffers from severe local model drift under non-IID data, significantly hindering global convergence. To address this, we propose the first FL framework incorporating Feedback Alignment (FA), introducing a zero-communication, parameter-free local training mechanism: during backpropagation, the global model weights are directly reused as the shared feedback matrix, thereby intrinsically aligning local gradients with the global objective and mitigating drift. We provide theoretical analysis proving that this mechanism improves both convergence rate and stability. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate consistent performance gains over mainstream algorithms—including FedAvg and FedProx—achieving accuracy improvements of 1.2–3.8%, faster convergence, and enhanced robustness to data heterogeneity.

Technology Category

Machine Learning: Distributed Machine Learning & Federated LearningSearch and Optimization: Local SearchComputer Vision: Large Vision Models

Application Category

User Modeling, Personalization and Recommendation: Federated recommendation systems and personalizationSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingGraph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Federated Learning (FL) enables collaborative training across multiple clients while preserving data privacy, yet it struggles with data heterogeneity, where clients' data are not distributed independently and identically (non-IID). This causes local drift, hindering global model convergence. To address this, we introduce Federated Learning with Feedback Alignment (FLFA), a novel framework that integrates feedback alignment into FL. FLFA uses the global model's weights as a shared feedback matrix during local training's backward pass, aligning local updates with the global model efficiently. This approach mitigates local drift with minimal additional computational cost and no extra communication overhead. Our theoretical analysis supports FLFA's design by showing how it alleviates local drift and demonstrates robust convergence for both local and global models. Empirical evaluations, including accuracy comparisons and measurements of local drift, further illustrate that FLFA can enhance other FL methods demonstrating its effectiveness.

Problem

• Addresses data heterogeneity in federated learning
• Mitigates local drift to improve global convergence
• Integrates feedback alignment with minimal computational overhead

Innovation

• Integrates feedback alignment into federated learning
• Uses global model weights as shared feedback matrix
• Mitigates local drift with minimal computational cost