Kernelized Edge Attention: Addressing Semantic Attention Blurring in Temporal Graph Neural Networks

2026-01-31

AI Summary

Existing temporal graph neural networks often conflate the semantics of node states and edge interactions within their attention mechanisms, struggling to distinguish between slowly evolving node dynamics and instantaneous edge events, which leads to semantic ambiguity. To address this, this work proposes KEAT, a novel approach that introduces, for the first time, a kernelized edge attention mechanism. By modulating edge features through continuous-time kernel functions—such as Laplacian, RBF, and learnable MLP kernels—KEAT explicitly decouples the roles of nodes and edges in temporal dynamics. The method is compatible with both Transformer and message-passing architectures, significantly enhancing the model’s capacity to capture fine-grained temporal dependencies and improving interpretability. On link prediction tasks, KEAT achieves up to 18% and 7% higher MRR than DyGFormer and TGN, respectively, demonstrating superior temporal awareness and predictive accuracy.

Technology Category

• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Temporal Planning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Temporal Graph Neural Networks (TGNNs) aim to capture the evolving structure and timing of interactions in dynamic graphs. Although many models incorporate time through encodings or architectural design, they often compute attention over entangled node and edge representations, failing to reflect their distinct temporal behaviors. Node embeddings evolve slowly as they aggregate long-term structural context, while edge features reflect transient, timestamped interactions (e.g. messages, trades, or transactions). This mismatch results in semantic attention blurring, where attention weights cannot distinguish between slowly drifting node states and rapidly changing, information-rich edge interactions. As a result, models struggle to capture fine-grained temporal dependencies and provide limited transparency into how temporal relevance is computed. This paper introduces KEAT (Kernelized Edge Attention for Temporal Graphs), a novel attention formulation that modulates edge features using a family of continuous-time kernels, including Laplacian, RBF, and learnable MLP variant. KEAT preserves the distinct roles of nodes and edges, and integrates seamlessly with both Transformer-style (e.g., DyGFormer) and message-passing (e.g., TGN) architectures. It achieves up to 18% MRR improvement over the recent DyGFormer and 7% over TGN on link prediction tasks, enabling more accurate, interpretable and temporally aware message passing in TGNNs.

Problem

• Temporal Graph Neural Networks
• Semantic Attention Blurring
• Edge Features
• Node Embeddings
• Temporal Dependencies

Innovation

• Temporal Graph Neural Networks
• Edge Attention
• Semantic Attention Blurring
• Continuous-time Kernels
• KEAT