Channel-Attentive Graph Neural Networks

2024-12-09

Industrial Conference on Data Mining

AI Summary

Deep stacking of Graph Neural Networks (GNNs) often leads to over-smoothing, causing node representation collapse and degraded generalization. To address this, we propose Channel-Aware Message Passing (CAM), the first GNN framework to incorporate channel-level attention into the message aggregation process, enabling adaptive selection of both informative neighbors and discriminative feature channels for information fusion. CAM further integrates neighborhood-adaptive weighting and heterogeneous graph structure modeling to mitigate over-smoothing and enhance semantic discrimination. Extensive experiments on multiple benchmark datasets demonstrate that CAM consistently outperforms state-of-the-art GNNs, achieving new SOTA performance—particularly on highly heterogeneous graph tasks. Moreover, CAM exhibits superior robustness against input noise and structural perturbations, validating its effectiveness in preserving fine-grained structural and semantic distinctions.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Graph Neural Networks (GNNs) set the state-of-the-art in representation learning for graph-structured data. They are used in many domains, from online social networks to complex molecules. Most GNNs leverage the message-passing paradigm and achieve strong performances on various tasks. However, the message-passing mechanism used in most models suffers from over-smoothing as a GNN's depth increases. The over-smoothing degrades GNN's performance due to the increased similarity between the representations of unrelated nodes. This study proposes an adaptive channel-wise message-passing approach to alleviate the over-smoothing. The proposed model, Channel-Attentive GNN, learns how to attend to neighboring nodes and their feature channels. Thus, much diverse information can be transferred between nodes during message-passing. Experiments with widely used benchmark datasets show that the proposed model is more resistant to over-smoothing than baselines and achieves state-of-the-art performances for various graphs with strong heterophily. Our code is at https://github.com/ALLab-Boun/CHAT-GNN.

Problem

• Addresses over-smoothing in deep Graph Neural Networks.
• Proposes adaptive channel-wise message-passing for diverse information transfer.
• Improves performance on graphs with strong heterophily.

Innovation

• Adaptive channel-wise message-passing approach
• Channel-Attentive GNN for diverse information transfer
• Resistant to over-smoothing in deep GNNs