TemporalGraphLLM: Temporal Graph Neural Networks with Large Language Models for Dynamic Text-Attributed Graphs

2026-09-25

AI Summary

This study addresses the challenge of synergistically modeling structural evolution and semantic reasoning in dynamic textual attributed graphs, where temporal graph neural networks (TGNNs) and large language models (LLMs) struggle to collaborate effectively. To this end, we propose TemporalGraphLLM, a framework that introduces a novel mechanism for seamlessly integrating arbitrary TGNNs into LLMs. By leveraging graph-temporal-aware instruction tuning and GNN embedding injection—where graph embeddings replace dedicated tokens—the proposed method effectively fuses structural dynamics with deep semantic understanding. Extensive experiments demonstrate that TemporalGraphLLM achieves state-of-the-art performance across edge classification, link prediction, and edge-based text generation tasks, validating the substantial potential of collaborative learning between TGNNs and LLMs.

Abstract

Dynamic text-attributed graphs (DTAGs), where nodes, edges, and textual attributes evolve over time, are crucial in applications such as social networks, citation graphs, and knowledge graphs. However, existing approaches struggle to jointly model the temporal evolution of graph structures and the semantic richness of textual attributes. While Temporal Graph Neural Networks (TGNNs) capture evolving node relationships, they often lack contextual text reasoning. Conversely, Large Language Models (LLMs) excel in textual understanding but struggle with structured graph reasoning in temporal settings. To bridge this gap, we propose TemporalGraphLLM, a novel framework that can integrate any temporal GNN with an LLM for enhanced reasoning in DTAGs. Our approach fine-tunes LLMs using graph-time-aware instruction tuning and novel temporal GNNs injection to replace dedicated added tokens with graph embeddings. TemporalGraphLLM effectively leverages pretrained TGNNs within an LLM framework to achieve state-of-the-art performance on edge classification, link prediction, and edge-based text generation tasks. Extensive evaluation on real-world dynamic graph datasets demonstrates state-of-the-art performance. Our findings highlight the synergistic potential of LLMs and TGNNs, opening new directions for learning on evolving graphs.

Problem

• Dynamic Text-Attributed Graphs
• Temporal Graph Neural Networks
• Large Language Models
• Temporal Evolution
• Graph Reasoning

Innovation

• Temporal Graph Neural Networks
• Large Language Models
• Dynamic Text-Attributed Graphs
• Instruction Tuning
• Graph Embedding Injection