graph-based concept reasoning

Graph Neural Networks (GNNs) suffer from opaque decision-making, and existing eXplainable AI (XAI) methods face trade-offs among structural awareness, computational efficiency, and generalizability—post-hoc approaches require black-box access and incur high overhead, while self-explaining models struggle to balance accuracy and versatility. Method: We propose GraphXAI, the first multi-granularity XAI framework integrating structural analysis and concept-based reasoning for GNNs. It disentangles topological patterns from semantic concepts within internal GNN representations, enabling verifiable explanations of how graph structure influences predictions. GraphXAI operates without retraining or auxiliary models and supports plug-and-play deployment. Contribution/Results: Evaluated on node and graph classification tasks, GraphXAI significantly improves explanation fidelity (+12.7%) and inference efficiency (3.2× speedup) over baselines. Crucially, it demonstrates strong cross-architecture and cross-dataset generalizability, establishing a new standard for scalable, structure-aware GNN explanation.

This work addresses the interpretability of Graph Neural Networks (GNNs) and Graph Transformers, aiming to uncover information flow mechanisms between nodes and elucidate how learned structural representations relate to the original graph topology. To this end, we propose Attention Graphs—a method that aggregates multi-layer, multi-head self-attention matrices to construct an explicit information propagation graph, unifying the message-passing characterization across both model families. Key contributions include: (i) establishing the first formal mathematical equivalence between GNN message passing and Transformer self-attention; (ii) revealing markedly distinct information-flow patterns among high-performing models on heterogeneous graphs, despite comparable accuracy; and (iii) demonstrating that under fully connected attention, learned structures exhibit weak correlation with the original graph. Extensive experiments on benchmark datasets validate that Attention Graphs effectively expose model inductive biases. The implementation is publicly available to ensure reproducibility.

Large language models (LLMs) exhibit significant attention misalignment with ideal graph topological distributions, hindering their ability to capture structural node relationships. Method: We propose the “intermediate-state attention window” strategy—preserving global context during training while dynamically focusing inference on critical neighborhoods, thereby balancing efficiency and structural completeness. Our approach integrates attention visualization analysis, structured prompt engineering, dynamic window adaptation, and joint graph-text embedding evaluation. Contribution/Results: Experiments reveal that while LLMs comprehend graph-text semantics, they severely mismatch graph topology. Our method improves downstream task accuracy by 12.7% and substantially enhances generalization. This work constitutes the first systematic identification and mitigation of attention misalignment in LLM-based graph modeling, establishing a principled framework for aligning LLM attention with graph structural priors.

Conventional concept-based interpretability methods provide only local explanations, failing to capture global neural circuit dynamics in deep models. Method: We propose the first concept-driven framework for global neural circuit analysis, introducing a model-agnostic, hierarchical concept analysis pipeline that quantifies the emergence, interaction, and propagation of semantic concepts across layers; we further design BAGEL—a visualization platform that encodes concept–class relationships as structured knowledge graphs to diagnose spurious correlations and data biases. Contributions/Results: (1) First systematic, end-to-end tracking of high-level semantic concepts throughout deep model internals; (2) Discovery of decision-critical latent neural circuits and information flow patterns; (3) Effective identification of generalization failures induced by data bias, substantially improving explanation fidelity and debugging capability.

Existing graph Transformers suffer from limited effectiveness, poor scalability, and high preprocessing complexity, often failing to outperform simple GNNs. To address this, we propose the first pure-attention graph learning framework that treats edge sets—not nodes—as the fundamental modeling unit, eliminating conventional node-centric representations and hand-crafted message passing. Our method introduces vertically interleaved masked and standard self-attention encoders, coupled with attention-based pooling for end-to-end differentiable training. It requires no graph reconstruction or preprocessing, natively supports heterogeneous graphs and transfer learning. Evaluated across 70+ node- and graph-level benchmark tasks, our approach consistently surpasses tuned GNN baselines and state-of-the-art graph Transformers. It achieves new SOTA results on molecular graph classification, vision-based graph recognition, heterogeneous graph learning, and cross-domain transfer, while maintaining both high accuracy and linear scalability.

This work addresses the limited transparency of graph neural network (GNN) inference by proposing a novel graph convolution approach that performs message passing within a node-level concept space, thereby enabling the first purely concept-space-based graph convolution. The method integrates raw input features with learnable concept representations and introduces a hybrid edge-weighting mechanism that combines structural priors with attention to enhance interpretability of concept evolution throughout the message-passing process. Experimental results demonstrate that the proposed approach achieves competitive task accuracy while significantly improving the understanding of GNNs’ internal decision-making logic.

Standard graph attention networks struggle with unreliable node features and fixed sharpness in their attention distributions, which limits robustness in noisy environments. This work proposes a gated graph attention mechanism that employs learnable gates to filter out unreliable features or messages and introduces a learnable temperature parameter to dynamically adjust the sharpness of the attention distribution. By doing so, the method significantly enhances robustness against both feature perturbations and global noise while preserving model expressiveness. Experimental results demonstrate that the proposed model consistently outperforms baseline approaches on both homophilic and heterophilic graph benchmarks and exhibits markedly improved robustness under various noise conditions.

This study investigates whether the attention matrices of Graph Transformers converge to a stable limit as the number of nodes grows. Grounded in dense graph limit theory, this work treats attention as samples from an underlying kernel function and introduces the concept of the "attention graphon." By employing cut-distance analysis, nonparametric estimation, and a block-averaging pipeline, it establishes assumption-free worst-case variance bounds alongside tighter regularity-based bounds. Empirical results confirm that attention converges to stable structures on specific datasets, validating the theoretical bounds and enabling cross-scale transferability. Ultimately, this research provides rigorous theoretical foundations for understanding the asymptotic behavior of large-scale Graph Transformers.

This study addresses the lack of a systematic survey on the integration of attention mechanisms with graph neural networks (GNNs), which has hindered a clear understanding of their developmental trajectory. To bridge this gap, the work proposes a novel two-level taxonomic framework that organizes the field both historically and architecturally. At the upper level, it delineates three chronological phases: Graph Recurrent Attention Networks, Graph Attention Networks, and Graph Transformers. The lower level systematically catalogs representative models within each phase and compares their key characteristics. Through comprehensive literature review and taxonomy-based analysis, the paper elucidates the evolutionary pathway of attention in GNNs, clarifies the strengths and limitations of existing approaches, identifies open challenges, and outlines promising future directions. An accompanying open-source repository is provided to foster ongoing community research.

Graph neural networks (GNNs) often suffer from semantic information loss during pooling operations in graph classification, which hinders their ability to provide interpretability at both subgraph and graph levels. To address this limitation, this work proposes the Subgraph Concept Network (SCN), which employs soft clustering of node concept embeddings to jointly and end-to-end distill semantic concepts at both subgraph and graph granularities. SCN is the first method to enable collaborative learning of multi-level concepts within GNNs, thereby overcoming the conventional reliance on node embeddings alone for interpretation. The approach achieves competitive graph classification performance while significantly enhancing model interpretability through explicit, hierarchical concept discovery.