One Model for One Graph: A New Perspective for Pretraining with Cross-domain Graphs

2024-11-30

arXiv.org

AI Summary

Weak generalization of existing Graph Neural Networks (GNNs) necessitates designing and training models from scratch for each graph dataset, hindering cross-domain transfer. To address this, we propose a “one-graph-one-model” cross-domain pretraining paradigm that abandons the pursuit of a single universal model. Instead, we construct a library of domain-specific expert models tailored to diverse graph domains and introduce a learnable, graph-level gating mechanism to dynamically route each new graph to its optimal expert model. This framework effectively avoids negative transfer and enables fine-grained, lightweight graph-level knowledge reuse. Extensive experiments on link prediction and node classification demonstrate that our approach significantly outperforms state-of-the-art GNNs and cross-domain baselines, validating its superior generalization capability and efficient adaptation to unseen graph domains.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Graph Neural Networks (GNNs) have emerged as a powerful tool to capture intricate network patterns, achieving success across different domains. However, existing GNNs require careful domain-specific architecture designs and training from scratch on each dataset, leading to an expertise-intensive process with difficulty in generalizing across graphs from different domains. Therefore, it can be hard for practitioners to infer which GNN model can generalize well to graphs from their domains. To address this challenge, we propose a novel cross-domain pretraining framework,"one model for one graph,"which overcomes the limitations of previous approaches that failed to use a single GNN to capture diverse graph patterns across domains with significant gaps. Specifically, we pretrain a bank of expert models, with each one corresponding to a specific dataset. When inferring to a new graph, gating functions choose a subset of experts to effectively integrate prior model knowledge while avoiding negative transfer. Extensive experiments consistently demonstrate the superiority of our proposed method on both link prediction and node classification tasks.

Problem

• Overcoming domain-specific GNN design limitations
• Enabling cross-domain graph generalization
• Avoiding negative transfer in graph pretraining

Innovation

• Cross-domain pretraining framework for GNNs
• Bank of expert models for diverse graphs
• Gating functions to integrate expert knowledge