Low-Rank Graph Contrastive Learning for Node Classification

2024-02-14

arXiv.org

AI Summary

Noisy real-world graph data severely degrades the node classification performance of Graph Neural Networks (GNNs). To address this, we propose Low-Rank Graph Contrastive Learning (LR-GCL), a framework that jointly optimizes structural robustness—enforced via low-frequency priors—and semantic discriminability by integrating low-rank regularization into prototype-based contrastive learning. We theoretically derive the first tight generalization bound for graph contrastive learning, revealing how low-rank modeling fundamentally enhances both generalization and noise resilience. LR-GCL adopts a transductive learning paradigm and employs a linear classifier for efficient deployment. Extensive experiments on multiple benchmark datasets demonstrate an average accuracy improvement of 2.3% over state-of-the-art methods. Moreover, LR-GCL exhibits superior robustness under label noise and edge perturbations, validating its enhanced generalization capability and structural stability.

Technology Category

Machine Learning: Graph-based Machine LearningNatural Language Processing: Safety and RobustnessComputer Vision: Adversarial Attacks & Robustness

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Graph Neural Networks (GNNs) have been widely used to learn node representations and with outstanding performance on various tasks such as node classification. However, noise, which inevitably exists in real-world graph data, would considerably degrade the performance of GNNs revealed by recent studies. In this work, we propose a novel and robust GNN encoder, Low-Rank Graph Contrastive Learning (LR-GCL). Our method performs transductive node classification in two steps. First, a low-rank GCL encoder named LR-GCL is trained by prototypical contrastive learning with low-rank regularization. Next, using the features produced by LR-GCL, a linear transductive classification algorithm is used to classify the unlabeled nodes in the graph. Our LR-GCL is inspired by the low frequency property of the graph data and its labels, and it is also theoretically motivated by our sharp generalization bound for transductive learning. To the best of our knowledge, our theoretical result is among the first to theoretically demonstrate the advantage of low-rank learning in graph contrastive learning supported by strong empirical performance. Extensive experiments on public benchmarks demonstrate the superior performance of LR-GCL and the robustness of the learned node representations. The code of LR-GCL is available at url{https://anonymous.4open.science/r/Low-Rank_Graph_Contrastive_Learning-64A6/}.

Problem

• Improves noisy node classification in graph data
• Introduces low-rank regularization for robust learning
• Enhances performance with low-rank attention layer

Innovation

• Graph Contrastive Learning with Low-Rank Regularization
• Two-stage transductive learning framework
• LR-Attention layer for reduced kernel complexity