Optimizing Federated Learning Using Remote Embeddings for Graph Neural Networks

2025-06-14

European Conference on Parallel Processing

AI Summary

To address the high communication overhead and slow convergence in federated graph neural networks (FedGNNs) caused by frequent remote embedding server interactions, this paper proposes OpES—a novel optimization framework. Methodologically, OpES introduces two key innovations: (1) a pipeline-based overlap mechanism that concurrently executes embedding push operations and local GNN training, enabling computation-communication parallelism; and (2) a lightweight remote neighborhood pruning strategy that significantly reduces transmission volume while maintaining bounded accuracy loss. Extensive experiments on large, dense graphs—including Reddit and Amazon Products—demonstrate that OpES achieves up to 2× faster convergence and up to 20% higher accuracy compared to state-of-the-art embedded federated learning approaches. By alleviating the communication bottleneck, OpES enables efficient and scalable FedGNN training without compromising model utility.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Graph Neural Networks (GNNs) have experienced rapid advancements in recent years due to their ability to learn meaningful representations from graph data structures. Federated Learning (FL) has emerged as a viable machine learning approach for training a shared model on decentralized data, addressing privacy concerns while leveraging parallelism. Existing methods that address the unique requirements of federated GNN training using remote embeddings to enhance convergence accuracy are limited by their diminished performance due to large communication costs with a shared embedding server. In this paper, we present OpES, an optimized federated GNN training framework that uses remote neighbourhood pruning, and overlaps pushing of embeddings to the server with local training to reduce the network costs and training time. The modest drop in per-round accuracy due to pre-emptive push of embeddings is out-stripped by the reduction in per-round training time for large and dense graphs like Reddit and Products, converging up to $approx2 imes$ faster than the state-of-the-art technique using an embedding server and giving up to $20%$ better accuracy than vanilla federated GNN learning.

Problem

• Reducing communication costs in federated GNN training
• Improving convergence speed for large dense graphs
• Enhancing accuracy in federated learning with remote embeddings

Innovation

• Optimizes federated GNN training with remote embeddings
• Uses remote neighborhood pruning to reduce costs
• Overlaps embedding push with local training