graph attention networks

To address the limited modeling capacity of Graph Neural Networks (GNNs) on heterogeneous graphs, this paper proposes Hetero-GAT+LPE, a heterogeneous graph attention network augmented with full-spectrum Laplacian positional encoding. Our method is the first to incorporate full-spectrum Laplacian eigenvectors as positional encodings into heterogeneous graph attention mechanisms, jointly capturing both absolute and relative structural positions of nodes—thereby mitigating the insufficient coupling between semantic and topological information inherent in heterogeneous graphs. Extensive experiments on multiple standard heterogeneous graph benchmarks demonstrate that Hetero-GAT+LPE consistently outperforms state-of-the-art GNNs on node classification and link prediction tasks, achieving average accuracy gains of 3.2%–5.7%. These results empirically validate the critical contribution of structural positional priors to representation learning on heterogeneous graphs.

Existing graph Transformers suffer from limited effectiveness, poor scalability, and high preprocessing complexity, often failing to outperform simple GNNs. To address this, we propose the first pure-attention graph learning framework that treats edge sets—not nodes—as the fundamental modeling unit, eliminating conventional node-centric representations and hand-crafted message passing. Our method introduces vertically interleaved masked and standard self-attention encoders, coupled with attention-based pooling for end-to-end differentiable training. It requires no graph reconstruction or preprocessing, natively supports heterogeneous graphs and transfer learning. Evaluated across 70+ node- and graph-level benchmark tasks, our approach consistently surpasses tuned GNN baselines and state-of-the-art graph Transformers. It achieves new SOTA results on molecular graph classification, vision-based graph recognition, heterogeneous graph learning, and cross-domain transfer, while maintaining both high accuracy and linear scalability.

This work investigates the fundamental limits of Graph Attention Networks (GATs) for node classification. Building upon the Contextual Stochastic Block Model (CSBM), we theoretically characterize GAT’s performance dependence on the relative magnitudes of structural and feature noise: GAT strictly outperforms GCN when structural noise dominates, but not necessarily otherwise. We establish the first rigorous signal-to-noise ratio (SNR) condition under which multi-layer GAT achieves perfect classification—improving the known lower bound from ω(√log n) to ω(√log n / ∛n). Furthermore, we elucidate GAT’s intrinsic mechanism for mitigating GCN’s oversmoothing via adaptive neighborhood aggregation. Our theoretical findings are validated empirically on both synthetic and real-world graphs, demonstrating that multi-layer GAT attains optimal classification under significantly milder SNR requirements than GCN. This work provides foundational theoretical insights for principled design and analysis of graph neural networks.

This work addresses the limited representational capacity of Graph Neural Networks (GNNs) when node features reside in uncountable spaces—e.g., continuous or infinite-dimensional domains. To overcome restrictive assumptions of countable features and strict injectivity, we propose Soft Isomorphism-aware Relational Graph Convolutional Networks (SIR-GCN). SIR-GCN replaces conventional discrete feature handling with pseudo-metric space modeling and soft injective functions, enabling context-aware, anisotropic, and dynamic message passing. We theoretically establish that SIR-GCN is a strict generalization of classical GNNs. Empirically, SIR-GCN achieves state-of-the-art performance on both node classification and graph-level property prediction tasks across synthetic benchmarks and standard datasets—including Cora, PPI, and ZINC—demonstrating substantially improved modeling capability and generalization for uncountable feature spaces.

Existing knowledge graph reasoning methods suffer from limited performance in entity classification and link prediction due to static neighbor aggregation and insufficient semantic modeling. To address this, we propose a dynamic representation learning framework that integrates attention mechanisms with graph convolutional networks (GCNs). Our key contributions are: (1) the first incorporation of attention mechanisms into every GCN layer to enable relation-aware, adaptive neighbor weighting during aggregation; and (2) a guided node representation learning paradigm based on entity similarity, which jointly models attribute and relational information to enhance implicit semantic capture. Extensive experiments on standard knowledge graph benchmarks demonstrate that our method achieves an average accuracy improvement of over 5.2% compared to state-of-the-art GNNs and embedding models on both entity classification and link prediction tasks. The results confirm significant gains in fine-grained entity representation quality and reasoning generalizability.

This work proposes a lightweight, data-driven document graph representation to overcome the limitations of traditional NLP systems that treat documents as linear sequences and struggle to model long-range dependencies and global structure. The approach automatically constructs a sentence-level graph using a dynamic sliding-window attention mechanism, effectively capturing local and medium-range semantic dependencies while preserving holistic document relationships. This graph is then processed by a Graph Attention Network (GAT) for downstream tasks. Evaluated on document classification benchmarks, the method achieves competitive performance with lower computational overhead and also demonstrates promising results in extractive summarization, highlighting its versatility and efficiency.

Traditional graph neural networks (GNNs) perform message passing solely through pairwise interactions between a central node and each individual neighbor, neglecting the holistic contextual information of the local neighborhood—thereby limiting their capacity to model complex neighborhood relationships. To address this, we formally define the concept of *neighborhood contextualization* and propose the Neighborhood Contextualized Message Passing (NCMP) framework, which transcends the conventional pairwise passing paradigm. NCMP introduces context-aware message generation via attention mechanisms and incorporates soft isomorphic neighborhood aggregation, leading to the SINC-GCN model. Evaluated on synthetic node classification tasks, SINC-GCN achieves substantial improvements in representation quality and classification accuracy. These results empirically validate both the effectiveness and necessity of explicitly modeling neighborhood context in graph representation learning.

This work addresses the limitation of existing graph Transformers, which rely on a single-token paradigm for graph-level representation and consequently fail to fully exploit the sequence modeling capacity of self-attention, often reducing to a weighted sum of node features. To overcome this, the authors propose a sequential graph tokenization paradigm that transforms node information into a sequence of tokens equipped with positional encodings. By stacking self-attention layers, the model captures complex dependencies among tokens, thereby unlocking the Transformer’s ability to model global structural information in graphs. This approach transcends the constraints of the conventional single-token framework and achieves state-of-the-art performance across multiple graph-level benchmark tasks. Ablation studies further confirm the effectiveness of each proposed component.

This work elucidates the theoretical mechanisms underlying the superior performance of Graph Transformers over conventional Graph Convolutional Networks in node-level prediction tasks, particularly their ability to mitigate oversmoothing. By analyzing the Neural Network Gaussian Process (NNGP) limit under infinite width and infinite attention heads, the authors derive inter-layer kernels for nodes and edges that characterize how node features and graph structure propagate through the attention mechanism. For the first time from a Gaussian process perspective, they formally demonstrate that Graph Transformers structurally preserve community information and maintain discriminative deep node representations. The proposed kernel design, which integrates positional encodings with informative priors, is empirically validated on both synthetic and real-world graph datasets, yielding significant performance gains in deep architectures.

Standard graph attention networks struggle with unreliable node features and fixed sharpness in their attention distributions, which limits robustness in noisy environments. This work proposes a gated graph attention mechanism that employs learnable gates to filter out unreliable features or messages and introduces a learnable temperature parameter to dynamically adjust the sharpness of the attention distribution. By doing so, the method significantly enhances robustness against both feature perturbations and global noise while preserving model expressiveness. Experimental results demonstrate that the proposed model consistently outperforms baseline approaches on both homophilic and heterophilic graph benchmarks and exhibits markedly improved robustness under various noise conditions.