ERAlign: Energy-based Representation Alignment of GNNs and LLMs on Text-attributed Graphs

2026-06-09

AI Summary

This work addresses the challenge of misaligned representations between graph neural networks (GNNs) and large language models (LLMs) in text-attributed graphs, which leads to representation drift and limited generalization. To resolve this, the authors propose an energy-driven, fine-grained alignment mechanism that maps GNN-encoded graph structures and LLM-generated text embeddings into a shared latent space via an energy-based model. Distributional consistency is achieved through layer-wise alignment coupled with energy minimization. The method introduces an energy discrepancy metric to reduce sampling costs while offering theoretical guarantees on training efficiency and fidelity of the energy landscape. Evaluated on eight text-attributed graph benchmarks, the approach achieves state-of-the-art performance and significantly enhances weakly supervised learning and cross-task transferability.

Abstract

Text-attributed Graphs (TAGs) incorporate textual node attributes with graph structures to describe rich relational semantics. Recent efforts to integrate Graph Neural Networks (GNNs) and Large Language Models (LLMs) have shown promise for learning on TAGs, yet achieving well-aligned representations remains challenging. Prior studies largely rely on heuristics that perform coarse-grained matching. They lack sufficient constraints and ignore distributional alignment, leading to representation drift and limited generalization. Building on Energy-based Models (EBMs), we propose an Energy-based Representation Alignment (ERAlign) framework that projects GNN-encoded graph structure and LLM-derived text embeddings in a shared latent space to achieve distribution consistency. Concretely, layer-wise alignment is quantified by a distance metric and optimized via an EBM objective. By decreasing energy values, our framework yields well-aligned representations for downstream tasks. During training, we introduce Energy Discrepancy (ED) to avoid high sampling costs associated with intractable normalization. ED also carries theoretical guarantees of higher training efficiency and reduced energy landscape distortion. Empirical evaluations on eight TAG datasets demonstrate that ERAlign obtains state-of-the-art performance across varying levels of supervision and cross-task transfer scenarios.

Problem

• Text-attributed Graphs
• Representation Alignment
• Graph Neural Networks
• Large Language Models
• Distribution Alignment

Innovation

• Energy-based Model
• Representation Alignment
• Text-attributed Graphs
• Graph Neural Networks
• Large Language Models