🤖 AI Summary
Existing graph neural network pre-training methods primarily target homogeneous graphs and overlook semantic mismatch—a prevalent issue in heterogeneous graphs wherein raw data exhibits a semantic gap relative to ideal, transfer-rich representations.
Method: We propose the first dual-aware pre-training framework for large-scale heterogeneous graphs. It jointly models heterogeneous topology via structure-aware pretext tasks and constructs semantic neighborhood perturbation subspaces through semantic-aware tasks, explicitly mitigating semantic mismatch in a self-supervised manner. The framework integrates heterogeneous structural modeling, semantic neighbor discovery, and perturbation subspace construction.
Contribution/Results: Evaluated on multiple real-world million-scale heterogeneous graph datasets, our method consistently outperforms state-of-the-art approaches, yielding average improvements of 3.2–5.8 percentage points across downstream tasks. It significantly enhances model generalizability and cross-task knowledge transfer capability.
📝 Abstract
In recent years, graph neural networks (GNNs) have facilitated the development of graph data mining. However, training GNNs requires sufficient labeled task-specific data, which is expensive and sometimes unavailable. To be less dependent on labeled data, recent studies propose to pre-train GNNs in a self-supervised manner and then apply the pre-trained GNNs to downstream tasks with limited labeled data. However, most existing methods are designed solely for homogeneous graphs (real-world graphs are mostly heterogeneous) and do not consider semantic mismatch (the semantic difference between the original data and the ideal data containing more transferable semantic information). In this paper, we propose an effective framework to pre-train GNNs on the large-scale heterogeneous graph. We first design a structure-aware pre-training task, which aims to capture structural properties in heterogeneous graphs. Then, we design a semantic-aware pre-training task to tackle the mismatch. Specifically, we construct a perturbation subspace composed of semantic neighbors to help deal with the semantic mismatch. Semantic neighbors make the model focus more on the general knowledge in the semantic space, which in turn assists the model in learning knowledge with better transferability. Finally, extensive experiments are conducted on real-world large-scale heterogeneous graphs to demonstrate the superiority of the proposed method over state-of-the-art baselines. Code available at https://github.com/sunshy-1/PHE.