MAWIFlow Benchmark: Realistic Flow-Based Evaluation for Network Intrusion Detection

2025-06-20

AI Summary

Existing NIDS benchmarks predominantly rely on synthetic traffic, failing to capture statistical variability and temporal drift inherent in real-world networks—leading to distorted model evaluation. To address this, we propose MAWIFlow: the first flow-level intrusion detection benchmark built exclusively on real backbone network traffic collected across three years (2011, 2016, 2021). MAWIFlow explicitly models temporal drift while preserving original ground-truth labels. We publicly release a standardized preprocessing pipeline, a reproducible PCAP-to-flow conversion tool, and temporally aligned multi-year datasets. Experimental results reveal that conventional tree-based models suffer over 40% performance degradation when evaluated across years, whereas CNN-BiLSTM maintains stable accuracy—demonstrating the critical necessity of long-term, realistic temporal benchmarks for assessing model generalization. This work pioneers the systematic integration of longitudinally evolving real-world network traffic into NIDS benchmark design.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Data Mining & Knowledge Management: Data Stream Mining
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Benchmark datasets for network intrusion detection commonly rely on synthetically generated traffic, which fails to reflect the statistical variability and temporal drift encountered in operational environments. This paper introduces MAWIFlow, a flow-based benchmark derived from the MAWILAB v1.1 dataset, designed to enable realistic and reproducible evaluation of anomaly detection methods. A reproducible preprocessing pipeline is presented that transforms raw packet captures into flow representations conforming to the CICFlowMeter format, while preserving MAWILab's original anomaly labels. The resulting datasets comprise temporally distinct samples from January 2011, 2016, and 2021, drawn from trans-Pacific backbone traffic. To establish reference baselines, traditional machine learning methods, including Decision Trees, Random Forests, XGBoost, and Logistic Regression, are compared to a deep learning model based on a CNN-BiLSTM architecture. Empirical results demonstrate that tree-based classifiers perform well on temporally static data but experience significant performance degradation over time. In contrast, the CNN-BiLSTM model maintains better performance, thus showing improved generalization. These findings underscore the limitations of synthetic benchmarks and static models, and motivate the adoption of realistic datasets with explicit temporal structure. All datasets, pipeline code, and model implementations are made publicly available to foster transparency and reproducibility.

Problem

• Realistic flow-based benchmark for intrusion detection evaluation
• Addressing limitations of synthetic traffic in anomaly detection
• Evaluating model performance degradation over temporal data shifts

Innovation

• MAWIFlow benchmark from real traffic data
• Reproducible preprocessing pipeline for flow conversion
• CNN-BiLSTM model for improved generalization