SENSOR: A Cost-Efficient Open-Source Flow Monitoring Platform

2025-07-21

AI Summary

High-performance networks require low-cost, high-fidelity traffic monitoring solutions that avoid the accuracy loss inherent in sampling-based approaches. Method: This paper designs and deploys a fully open-source, sampling-free distributed IPFIX monitoring platform. Built on lightweight open-source components—including goflow2, Kafka, Prometheus, and TimescaleDB—the system adopts a hierarchical edge-collection–central-aggregation architecture, enabling end-to-end real-time flow data acquisition, transmission, and storage across the University of Tübingen’s campus network. Contribution/Results: The platform guarantees 100% flow record completeness—unlike sampled alternatives—and reduces hardware costs by over 80% compared to commercial solutions, while supporting horizontal scalability. Empirical evaluation demonstrates sustained line-rate processing of 10 Gbps full-link traffic, handling over one billion flow records daily. It exhibits strong robustness and reproducibility, establishing a deployable, open-source monitoring paradigm for research and educational networks.

Technology Category

• Planning, Routing, and Scheduling: Plan Execution and Monitoring
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Multiagent Systems: Agent Communication

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Web measurements

Abstract

This paper presents a cost-effective and distributed flow monitoring platform for collecting unsampled IPFIX data exclusively using open-source tools, which is implemented at the University of Tübingen. An overview of all tools is given and their use is explained.

Problem

• Develops a cost-efficient flow monitoring platform
• Collects unsampled IPFIX data using open-source tools
• Implements distributed monitoring at University of Tübingen

Innovation

• Cost-effective open-source flow monitoring
• Distributed unsampled IPFIX collection
• Implemented with open-source tools