GraspLLM: Towards Zero-Shot Generalization on Text-Attributed Graphs with LLMs

2026-06-10

AI Summary

Existing methods struggle to achieve zero-shot generalization across datasets and tasks on Text-Attributed Graphs (TAGs) and inadequately model transferable structural patterns. This work proposes a novel framework that integrates the semantic understanding of large language models (LLMs) with graph structure awareness. By constructing a unified semantic embedding space and employing motif-aware contrastive learning, the approach captures dataset-agnostic structural information. Furthermore, it introduces an alignment mechanism between contextually optimal subgraphs and the LLM token space. Notably, the method requires no fine-tuning of off-the-shelf text encoders and achieves, for the first time, strong zero-shot cross-domain generalization with LLMs on TAGs. It substantially outperforms existing LLM-based approaches across multiple cross-domain benchmarks, demonstrating exceptional performance particularly in zero-shot settings.

Abstract

Research on Text-Attributed Graphs (TAGs) has gained significant attention recently due to its broad applications across various real-world data scenarios, such as citation networks, e-commerce platforms, social media, and web pages. Inspired by the remarkable semantic understanding ability of Large Language Models (LLMs), there have been numerous attempts to integrate LLMs into TAGs. However, existing methods still struggle to generalize across diverse graphs and tasks, and their ability to capture transferable graph structural patterns remains limited. To address this, we introduce the GraspLLM, a framework that combines Graph structural comprehension with semantic understanding prowess of LLMs to enhance the cross-dataset and cross-task generalizability. Specifically, we represent node texts from different graphs in a unified semantic space with a frozen general embedding model, on top of which we perform motif-aware contrastive learning across multiple motif-induced adjacency matrices to extract dataset-agnostic structural information. Then, with our proposed optimal contextual subgraph, we extract the most contextually relevant subgraph for each target node and align these subgraphs to the token space of LLM via an alignment projector. Extensive experiments on TAG benchmark datasets spanning diverse domains reveal that GraspLLM consistently outperforms previous LLM-based methods for TAGs, especially in zero-shot scenarios, highlighting its strong generalizability across different datasets and tasks. Our code is available at https://github.com/Heinz217/GraspLLM.

Problem

• Text-Attributed Graphs
• zero-shot generalization
• cross-dataset generalization
• cross-task generalization
• graph structural patterns

Innovation

• Text-Attributed Graphs
• Large Language Models
• Zero-Shot Generalization
• Motif-Aware Contrastive Learning
• Contextual Subgraph Alignment