Data-Driven Self-Supervised Graph Representation Learning

2024-12-24

European Conference on Artificial Intelligence

AI Summary

To address the poor generalizability, lack of theoretical foundation, and semantic distortion of graph augmentations in self-supervised graph learning, this paper proposes the first end-to-end learnable graph augmentation framework. Our method jointly models node feature perturbation and high-order topological view generation, enabling differentiable and optimization-friendly multi-view augmentation; it supports both homogeneous and heterogeneous graphs while avoiding heuristic, semantics-destructive operations (e.g., random edge/node dropping). Within a contrastive learning paradigm, we co-optimize the augmentation generator and representation encoder, incorporating a multi-view consistency loss. Extensive experiments demonstrate that our approach matches or surpasses state-of-the-art self-supervised methods on nine node classification and eight graph-level prediction benchmarks—achieving performance comparable to supervised or semi-supervised baselines.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Self-supervised graph representation learning (SSGRL) is a representation learning paradigm used to reduce or avoid manual labeling. An essential part of SSGRL is graph data augmentation. Existing methods usually rely on heuristics commonly identified through trial and error and are effective only within some application domains. Also, it is not clear why one heuristic is better than another. Moreover, recent studies have argued against some techniques (e.g., dropout: that can change the properties of molecular graphs or destroy relevant signals for graph-based document classification tasks). In this study, we propose a novel data-driven SSGRL approach that automatically learns a suitable graph augmentation from the signal encoded in the graph (i.e., the nodes' predictive feature and topological information). We propose two complementary approaches that produce learnable feature and topological augmentations. The former learns multi-view augmentation of node features, and the latter learns a high-order view of the topology. Moreover, the augmentations are jointly learned with the representation. Our approach is general that it can be applied to homogeneous and heterogeneous graphs. We perform extensive experiments on node classification (using nine homogeneous and heterogeneous datasets) and graph property prediction (using another eight datasets). The results show that the proposed method matches or outperforms the SOTA SSGRL baselines and performs similarly to semi-supervised methods. The anonymised source code is available at https://github.com/AhmedESamy/dsgrl/

Problem

• Self-Supervised Graph Learning
• Data Augmentation
• Theoretical Foundation

Innovation

• Self-supervised Learning
• Graph Data Augmentation
• Adaptive Graph Learning