Edge-Aware Curvature Modeling for Graph Understanding in Large Language Models

2026-06-04

AI Summary

Existing graph-aware large language models often neglect edge structure during graph-text alignment, leading to limited cross-modal information propagation and over-compression issues. This work proposes CureLLM, a novel framework that theoretically demonstrates for the first time that ignoring edge information results in suboptimal alignment. CureLLM introduces an innovative, training-free edge-aware textual prompting mechanism coupled with curvature-aware graph representation learning. By leveraging positively curved edges to guide message passing, the method effectively injects structural information into a frozen large language model. Evaluated on 11 real-world datasets, CureLLM significantly outperforms 20 baseline methods and achieves state-of-the-art performance in graph–text joint understanding tasks.

Abstract

Recently, graph-aware Large Language Models (LLMs) have shown promising capabilities in jointly modeling graph-structured data and textual information. Existing approaches typically employ a graph encoder and a frozen LLM to obtain node representations from graph and textual views, followed by node-level alignment to bridge the two modalities. However, such alignment mechanisms primarily focus on node information while overlooking edge-level structures, leading to suboptimal information propagation across views. In this work, we conduct a comprehensive theoretical analysis to uncover why node-level alignment is insufficient for aligning textual and graph representations. Specifically, we prove theoretically for the first time that neglecting edge information leads to suboptimal solutions and negatively curved edges induce bottlenecked information flow, giving rise to the over-squashing phenomenon between graph and textual views. To address the two challenges, we innovatively proposed a CureLLM framework of Curvature-enhanced Graph Representations for Large Language Model whose goal is to inject the signals of edge information into the existing LLMs. Specifically, CureLLM first introduces the training-free textual prompt mechanism to make the LLM model generate the output directly based on the edge-aware prompt without learnable parameter costs. Furthermore, a novel curvature-aware graph representation learning is designed to capture the edge structure information to enhance the downstream tasks, where the message passing between text and graph representations only depends on edges with positive curvature. Finally, we conduct evaluations with 20 different compared methods on 11 real world datasets from various domains and the experiment results demonstrate the superiority of our proposed CureLLM framework.

Problem

• graph-text alignment
• edge-aware modeling
• over-squashing
• curvature
• large language models

Innovation

• edge-aware curvature
• over-squashing
• graph-text alignment
• curvature-enhanced representation
• training-free prompting