🤖 AI Summary
Real-time detection of unknown attacks in dynamic network threat environments remains challenging, and conventional offline methods suffer from delayed retraining. Method: This paper investigates unsupervised streaming anomaly detection on process-level security event streams, leveraging the eBPF-collected BETH dataset. We systematically evaluate ten streaming learning algorithms under realistic OS behavioral streams, employing River/scikit-multiflow/creme frameworks. A tailored feature engineering pipeline incorporates process lifecycle semantics and temporal statistics, explicitly fusing event timing and multidimensional features. Evaluation adopts dual metrics: ROC-AUC and end-to-end latency. Contribution/Results: Hoeffding Tree augmented with ADWIN concept drift detection achieves state-of-the-art ROC-AUC (0.92) at millisecond-scale latency—significantly outperforming offline baselines. Our study is the first to empirically demonstrate the critical impact of temporal-feature fusion on detection accuracy and validates the effectiveness and practicality of streaming paradigms for near-real-time identification of previously unseen threats.
📝 Abstract
In modern world the importance of cybersecurity of various systems is increasing from year to year. The number of information security events generated by information security tools grows up with the development of the IT infrastructure. At the same time, the cyber threat landscape does not remain constant, and monitoring should take into account both already known attack indicators and those for which there are no signature rules in information security products of various classes yet. Detecting anomalies in large cybersecurity data streams is a complex task that, if properly addressed, can allow for timely response to atypical and previously unknown cyber threats. The possibilities of using of offline algorithms may be limited for a number of reasons related to the time of training and the frequency of retraining. Using stream learning algorithms for solving this task is capable of providing near-real-time data processing. This article examines the results of ten algorithms from three Python stream machine-learning libraries on BETH dataset with cybersecurity events, which contains information about the creation, cloning, and destruction of operating system processes collected using extended eBPF. ROC-AUC metric and total processing time of processing with these algorithms are presented. Several combinations of features and the order of events are considered. In conclusion, some mentions are given about the most promising algorithms and possible directions for further research are outlined.