Adaptive-GraphSketch: Real-Time Edge Anomaly Detection via Multi-Layer Tensor Sketching and Temporal Decay

2025-09-15

AI Summary

Dynamic graph edge anomaly detection faces challenges including poor scalability, lack of probabilistic interpretability, and insufficient adaptability to traffic surges—particularly in cybersecurity applications. To address these, we propose a lightweight streaming detection framework that integrates multi-layer tensor sketching with temporal decay, employing a conservative-update variant of Count-Min Sketch for compact modeling of high-frequency patterns. We further introduce a Bayesian anomaly scoring mechanism coupled with an Exponentially Weighted Moving Average (EWMA)-based adaptive threshold to enhance burst sensitivity and probabilistic interpretability. The method uses only ten hash functions, ensuring low memory overhead and high throughput. Evaluated on four real-world intrusion detection datasets, it significantly outperforms baselines including ANOEDGE-G/L and MIDAS-R: AUC improves by 6.5% on CIC-IDS2018 and 15.6% on CIC-DDoS2019; processing 20 million edges takes merely 3.4 seconds.

Technology Category

Data Mining & Knowledge Management: Anomaly/Outlier DetectionMachine Learning: Graph-based Machine LearningReasoning under Uncertainty: Graphical Models

Application Category

Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphsWeb Mining and Content Analysis: Content-based information diffusionSecurity and Privacy: Large-scale security measurements

Abstract

Anomaly detection in dynamic graphs is essential for identifying malicious activities, fraud, and unexpected behaviors in real-world systems such as cybersecurity and power grids. However, existing approaches struggle with scalability, probabilistic interpretability, and adaptability to evolving traffic patterns. In this paper, we propose ADAPTIVE-GRAPHSKETCH, a lightweight and scalable framework for real-time anomaly detection in streaming edge data. Our method integrates temporal multi-tensor sketching with Count-Min Sketch using Conservative Update (CMS-CU) to compactly track edge frequency patterns with bounded memory, while mitigating hash collision issues. We incorporate Bayesian inference for probabilistic anomaly scoring and apply Exponentially Weighted Moving Average (EWMA) for adaptive thresholding tuned to burst intensity. Extensive experiments on four real-world intrusion detection datasets demonstrate that ADAPTIVE-GRAPHSKETCH outperforms state-of-the-art baselines such as ANOEDGE-G/L, MIDAS-R, and F-FADE, achieving up to 6.5% AUC gain on CIC-IDS2018 and up to 15.6% on CIC-DDoS2019, while processing 20 million edges in under 3.4 seconds using only 10 hash functions. Our results show that ADAPTIVE-GRAPHSKETCH is practical and effective for fast, accurate anomaly detection in large-scale streaming graphs. Keywords: Anomaly Detection, Streaming, Real-time, Dynamic Graphs, Edge Streams, Tensor Sketching

Problem

• Real-time anomaly detection in streaming edge data
• Scalable framework for dynamic graph anomaly identification
• Adaptive probabilistic scoring with bounded memory usage

Innovation

• Multi-layer tensor sketching for edge tracking
• Bayesian inference for probabilistic anomaly scoring
• EWMA adaptive thresholding for burst intensity