FedAGHN: Personalized Federated Learning with Attentive Graph HyperNetworks

2025-01-24

AI Summary

In personalized federated learning (PFL), strong statistical heterogeneity across clients and dynamically evolving collaboration relationships pose significant challenges. To address these, this paper proposes an Adjustable Graph Hypernetwork (AGHN), which explicitly models a fine-grained client collaboration graph. AGHN employs a phase-aware attention mechanism to adaptively learn semantic similarities among clients and generate client-specific personalized initial models. By integrating graph neural networks with hypernetworks, AGHN enables dynamic, interpretable, and topology-aware personalized model aggregation. Extensive experiments on multiple heterogeneous benchmark datasets demonstrate that AGHN consistently outperforms state-of-the-art PFL methods in both accuracy and convergence stability. Visualization analyses further confirm that the learned collaboration graphs accurately capture dynamic inter-client synergies—revealing meaningful, evolving relational structures. This work establishes a novel paradigm for relational modeling in PFL, advancing both the expressiveness and interpretability of personalized aggregation mechanisms.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Multiagent Systems: Multiagent Learning

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Personalized Federated Learning (PFL) aims to address the statistical heterogeneity of data across clients by learning the personalized model for each client. Among various PFL approaches, the personalized aggregation-based approach conducts parameter aggregation in the server-side aggregation phase to generate personalized models, and focuses on learning appropriate collaborative relationships among clients for aggregation. However, the collaborative relationships vary in different scenarios and even at different stages of the FL process. To this end, we propose Personalized Federated Learning with Attentive Graph HyperNetworks (FedAGHN), which employs Attentive Graph HyperNetworks (AGHNs) to dynamically capture fine-grained collaborative relationships and generate client-specific personalized initial models. Specifically, AGHNs empower graphs to explicitly model the client-specific collaborative relationships, construct collaboration graphs, and introduce tunable attentive mechanism to derive the collaboration weights, so that the personalized initial models can be obtained by aggregating parameters over the collaboration graphs. Extensive experiments can demonstrate the superiority of FedAGHN. Moreover, a series of visualizations are presented to explore the effectiveness of collaboration graphs learned by FedAGHN.

Problem

• Personalized Federated Learning
• Data Heterogeneity
• Dynamic Collaboration

Innovation

• FedAGHN
• Attention-based Hypernetworks
• Personalized Federated Learning