Scalable and Efficient Joint Spiking Embedding Predictive Architecture for Large-Scale Dynamic Graphs

2026-07-20

AI Summary

This work addresses the high computational cost and poor scalability of existing self-supervised learning methods for dynamic graphs, which typically rely on edge-level reconstruction or graph augmentation. The authors propose SG-JEPA, a novel architecture that introduces spiking neural networks to large-scale dynamic graph representation learning for the first time. Leveraging a spatiotemporal predictive coding mechanism, SG-JEPA partitions nodes along the temporal dimension into context and target sets for mutual prediction. It employs spiking neurons to generate coarse-to-fine spike-count embeddings, flexibly accommodating varying computational constraints across downstream tasks. Notably, the method eliminates complex components such as negative sampling, graph augmentation, and edge reconstruction, enabling efficient training on a large-scale dynamic graph with 13 million edges. It achieves node classification performance on par with or superior to discriminative baselines while significantly improving training efficiency and memory scalability.

Abstract

Dynamic graph learning aims to capture evolving structural and semantic patterns in real-world systems, such as fraud detection and recommender systems. Due to the scarcity of labeled data in real-world dynamic graphs, recent studies have introduced generative or contrastive paradigms (e.g., masked graph autoencoders or graph contrastive learning) to generate task-agnostic graph embeddings. However, these methods typically rely on complex edge-level reconstruction objectives and tailored graph augmentation strategies. This incurs substantial computational overhead when scaling to large-scale dynamic graphs. In this paper, we propose SG-JEPA, a joint spiking embedding predictive architecture for large-scale dynamic graphs. In contrast to existing self-supervised methods, SG-JEPA partitions nodes into context and target sets along the temporal dimension to learn embeddings that are predictive of each other via additional spatial-temporal information. Furthermore, through encoding sequential inputs into coarse-to-fine spike count embeddings, spiking neurons enable SG-JEPA to adapt to the varying computational constraints of downstream tasks. Extensive experiments demonstrate that SG-JEPA achieves competitive or even superior performance over discriminative baselines on node classification, while effectively scaling to the dynamic graph with 13 million edges. SG-JEPA avoids the complex machinery (negative sampling, graph augmentations, edge-level reconstruction, etc.), resulting in superior training efficiency and memory scalability compared with prior self-supervised dynamic graph baselines.

Problem

• dynamic graph learning
• self-supervised learning
• scalability
• computational overhead
• graph embedding

Innovation

• spiking neural networks
• dynamic graph learning
• self-supervised learning
• scalable embedding
• temporal prediction