🤖 AI Summary
Masked feature reconstruction (MFR) in graph self-supervised learning suffers from weak discriminability and conceptual disconnection from contrastive learning. Method: This paper theoretically establishes, under reasonable assumptions, the objective-function equivalence between MFR and node-level graph contrastive learning (GCL). Building on this insight, we propose Contrastive Masked Feature Reconstruction (CMFR)—a unified framework that introduces a novel contrastive reconstruction paradigm: original and reconstructed features serve as positive pairs, while masked nodes act as negatives. CMFR integrates a context-aware encoder and a customized negative sampling strategy. Contribution/Results: On multiple benchmark datasets, CMFR consistently outperforms GraphMAE and GraphMAE2, achieving up to 3.82% absolute improvement in both node and graph classification tasks, setting a new state-of-the-art in graph self-supervised learning.
📝 Abstract
In the rapidly evolving field of self-supervised learning on graphs, generative and contrastive methodologies have emerged as two dominant approaches. Our study focuses on masked feature reconstruction (MFR), a generative technique where a model learns to restore the raw features of masked nodes in a self-supervised manner. We observe that both MFR and graph contrastive learning (GCL) aim to maximize agreement between similar elements. Building on this observation, we reveal a novel theoretical insight: under specific conditions, the objectives of MFR and node-level GCL converge, despite their distinct operational mechanisms. This theoretical connection suggests these approaches are complementary rather than fundamentally different, prompting us to explore their integration to enhance self-supervised learning on graphs. Our research presents Contrastive Masked Feature Reconstruction (CORE), a novel graph self-supervised learning framework that integrates contrastive learning into MFR. Specifically, we form positive pairs exclusively between the original and reconstructed features of masked nodes, encouraging the encoder to prioritize contextual information over the node's own features. Additionally, we leverage the masked nodes themselves as negative samples, combining MFR's reconstructive power with GCL's discriminative ability to better capture intrinsic graph structures. Empirically, our proposed framework CORE significantly outperforms MFR across node and graph classification tasks, demonstrating state-of-the-art results. In particular, CORE surpasses GraphMAE and GraphMAE2 by up to 2.80% and 3.72% on node classification tasks, and by up to 3.82% and 3.76% on graph classification tasks.