GraphEdit: Large Language Models for Graph Structure Learning

2024-02-23

arXiv.org

AI Summary

Existing graph structure learning (GSL) methods heavily rely on explicit ground-truth graph structure supervision, rendering them vulnerable to structural noise and sparsity. Method: This paper introduces large language models (LLMs) into GSL for the first time, proposing an end-to-end framework that operates without access to true graph topology. It employs graph-structure instruction tuning and node-relation serialization encoding to equip LLMs with the capacity to model global node dependencies and perform automatic edge denoising. Contribution/Results: The key innovation lies in eliminating strong reliance on prior graph structures, instead leveraging LLMs’ generalization and reasoning capabilities for globally consistent structural inference. Extensive experiments on multiple benchmark datasets demonstrate significant improvements over conventional GSL approaches—particularly under high-noise and low-connectivity regimes—where robustness gains are especially pronounced.

Technology Category

Machine Learning: Graph-based Machine LearningNatural Language Processing: (Large) Language ModelsPlanning, Routing, and Scheduling: Planning with Language Models

Application Category

Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphsSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsUser Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Graph Structure Learning (GSL) focuses on capturing intrinsic dependencies and interactions among nodes in graph-structured data by generating novel graph structures. Graph Neural Networks (GNNs) have emerged as promising GSL solutions, utilizing recursive message passing to encode node-wise inter-dependencies. However, many existing GSL methods heavily depend on explicit graph structural information as supervision signals, leaving them susceptible to challenges such as data noise and sparsity. In this work, we propose GraphEdit, an approach that leverages large language models (LLMs) to learn complex node relationships in graph-structured data. By enhancing the reasoning capabilities of LLMs through instruction-tuning over graph structures, we aim to overcome the limitations associated with explicit graph structural information and enhance the reliability of graph structure learning. Our approach not only effectively denoises noisy connections but also identifies node-wise dependencies from a global perspective, providing a comprehensive understanding of the graph structure. We conduct extensive experiments on multiple benchmark datasets to demonstrate the effectiveness and robustness of GraphEdit across various settings. We have made our model implementation available at: https://github.com/HKUDS/GraphEdit.

Problem

• Overcome limitations of explicit graph structural information
• Enhance reliability of graph structure learning
• Denoise noisy connections and identify node-wise dependencies

Innovation

• Leverages large language models for graph learning
• Enhances LLMs with instruction-tuning on graphs
• Denoises and identifies node dependencies globally