network traffic analysis

Designs and builds capture, parsing, and analysis tools and methods that collect and examine network packets, packet captures, network flows, and HTTP headers to monitor and measure network traffic, characterize endpoints and URLs, and detect transmitted content or metadata indicative of data leakage, unwanted communications, or protocol-level behavior.

networktrafficanalysis

Recent Skill Trend

Momentum and market value over time
Trending
Score
No comparison yet
-0.12
Oct 01, 2026Oct 01, 2026
Career
Value
No comparison yet
$192K/year
Oct 01, 2026Oct 01, 2026

Must-Read Papers

Most classic and influential ideas
View more

This work proposes a novel interactive analysis system centered on three-dimensional network topology to overcome the limitations of traditional PCAP analysis tools, which present data as linear lists and fail to reveal underlying communication structures. The system maps hosts, sessions, and protocols to nodes, edges, and visual clusters, respectively, and enables bidirectional synchronized filtering with the packet list. By adopting 3D space as the default view—implemented using Three.js—it intuitively encodes key features such as communication density, clustering structure, host centrality, and traffic volume through depth perception. Supporting parsing of PCAP/PCAPNG formats and decoding of over 90 protocols, the approach significantly enhances the observability of structural patterns in network traffic, facilitating efficient identification of anomalous communications, critical nodes, and protocol distributions.

interactive visualizationnetwork topologypacket analysis

Compact Data Structures for Network Telemetry

Nov 05, 2023
SL
Shir Landau Feibish
🏛️ The Open University of Israel | University of Maryland | Princeton University

Conventional network telemetry frameworks struggle to support fine-grained traffic measurement, performance diagnostics, and attack detection under stringent memory and computational constraints of high-speed network devices. Method: This paper proposes a lightweight, real-time online telemetry framework that systematically integrates compact data structures—including Bloom filter variants, Count-Min Sketch, and HyperLogLog—with streaming algorithms, hierarchical sampling, and P4-programmable data-plane co-design to comply with hardware limitations. Contribution/Results: Evaluated at line rate exceeding 100 Gbps, the framework reduces memory footprint by over 60% compared to state-of-the-art approaches while maintaining sub-1% flow frequency estimation error. It achieves an optimal trade-off among accuracy, throughput, and resource overhead, thereby significantly enhancing the feasibility and practicality of telemetry in high-bandwidth environments.

Compact data structures for traffic analysisHigh-speed network device limitationsTrade-offs between accuracy and overhead

This work proposes an end-to-end, reproducible supervised traffic flow classification framework that addresses the limitations of traditional port- or payload-based methods in the face of encrypted and increasingly diverse network traffic. The framework integrates practical considerations from real-world measurements, incorporating flow-based feature extraction, time-aware data splitting, leakage-proof experimental design, and interpretability analysis to mitigate common methodological pitfalls. Accompanied by an open-source Jupyter Notebook implementation, it provides a complete pipeline—from traffic capture and dataset construction to model training, evaluation, and deployment. Empirical validation on real-world encrypted traffic demonstrates the approach’s effectiveness, robustness, and practical deployability.

encrypted trafficflow-based classificationmachine learning

A Novel Approach to Network Traffic Analysis: the HERA tool

Jan 13, 2025
DP
Daniela Pinto
🏛️ Polytechnic of Porto | PORTIC

Existing network traffic analysis tools (e.g., CICFlowMeter) suffer from critical limitations in flow delineation, feature extraction, and label consistency, undermining the reliability and reproducibility of intrusion detection systems (IDS). To address these issues, we propose HERA—a lightweight, open-source, end-to-end traffic processing framework. HERA is the first tool to support configurable feature sets and fine-grained flow labeling, integrating NetFlow/IPFIX parsing, customizable feature engineering, and flexible label mapping. Implemented in Python, it natively supports standard datasets such as UNSW-NB15. Experimental evaluation on UNSW-NB15 demonstrates >99.8% flow generation accuracy and 100% label consistency across all flows. HERA significantly enhances traffic data fidelity, usability, and extensibility, thereby establishing a high-fidelity, reproducible foundation for IDS research and development.

Accuracy IssuesFeature Selection LimitationsNetwork Traffic Analysis

This work addresses the limitations of traditional passive network measurement, which primarily focuses on inbound traffic and struggles to detect stealthy internal anomalies. The paper presents the first systematic approach that leverages erroneous outbound traffic—such as unanswered requests and ICMP error messages—as a lightweight yet highly informative data source. By conducting large-scale passive monitoring and correlation analysis, the method effectively identifies misconfigurations, deprecated services, and potentially compromised hosts within internal networks. Deployed in large-scale operational environments, this technique has uncovered a variety of previously undetected internal anomalies, substantially enhancing visibility into and detection capabilities for internal threats.

erroneous outbound trafficICMP errorsinternal anomalies

Latest Papers

What's happening recently
View more

Traditional traffic analysis has become ineffective due to the widespread adoption of encryption and privacy-enhancing technologies. Existing machine learning approaches often rely on protocol-specific features, require large amounts of labeled data, and exhibit poor generalization across domains. To address these limitations, this work proposes the first purely metadata-driven, protocol-agnostic framework for encrypted traffic analysis. By modeling network flows as multivariate time series and integrating meta-learning, embedding optimization, and self-attention mechanisms, the framework enables rapid cross-scenario adaptation under few-shot conditions. Evaluated across nine public datasets on tasks including application identification, VPN traffic classification, IoT device fingerprinting, and attack detection, the method consistently outperforms state-of-the-art approaches, demonstrating strong generality, robustness, and practical utility.

encrypted traffic analysisfew-shot adaptationheterogeneous network environments

This work addresses the limitations of existing cybersecurity datasets, which are predominantly static and ill-suited for enabling controllable replay and traceability in heterogeneous, multi-protocol environments. To overcome this, the authors propose a scenario-oriented, container-native testing platform that leverages declarative configuration to parameterize the generation of both adversarial and benign network traffic, log collection, and dataset integration. The platform encapsulates 60 attack scenarios, nine target services, and benign traffic generators within single-purpose containers and integrates them into an automated pipeline for feature extraction and experimental execution. Designed with reproducibility, auditability, and extensibility in mind, the framework significantly reduces operational bias and supports fully traceable, reproducible experiments in complex settings such as IoT and IIoT networks.

cybersecurity experimentationdataset generationmulti-protocol environments

This study addresses the security risks in modern web applications—such as cache poisoning and supply chain attacks—stemming from redundant HTTP API requests, missing cache headers, high load, and excessive reliance on third-party services, for which systematic evaluation methods are lacking. The authors present the first empirical baseline of HTTP API quality across diverse production websites, collecting 108 HAR traces from 18 sites using Playwright automation. They design eight heuristic-based anti-pattern detectors to quantify API quality on a 0–100 scale and correlate it with security implications. Findings reveal that minimal server-rendered sites achieve a perfect score of 100, while content-heavy commercial sites score as low as 56.8; 67% of sites exhibit redundant requests or cache misconfigurations, and 72% have over 20% third-party requests, with one page issuing up to 2,684 such calls. The open-sourced framework enables reproducible, systematic linkage between performance anti-patterns and security risks.

anti-patternsHTTP APInetwork layer quality

This study addresses the challenge of large-scale unsolicited Internet traffic targeting Internet of Things (IoT) devices and its associated security threats by proposing a lightweight monitoring approach that operates without payload inspection. Leveraging data collected via network telescopes, the method integrates privacy-preserving metadata analysis, behavioral heuristics, and Shannon entropy measurements to effectively identify coordinated scanning and backscatter activities. The findings reveal that the top 1% of source IP addresses generate over 81% of the observed traffic, with Telnet ports (23/2323) dominating the activity—evidence of highly concentrated, synchronized, and multi-vector reconnaissance campaigns. This work provides a scalable and practical analytical framework for enhancing large-scale IoT threat situational awareness.

IoT security threatsnetwork telescopesreconnaissance campaigns

Hot Scholars

ZS

Zubair Shafiq

University of California, Davis
Online PrivacyInternet MeasurementTech Policy
QW

Qin Wang

ETH Zurich
Domain AdaptationComputer Vision
YV

Yash Vekaria

PhD Researcher, University of California at Davis
PrivacySecurityInternet MeasurementsLLMs
MW

Matthias Wählisch

Professor and Chair of Distributed and Networked Systems, TU Dresden, BI Research Fellow
Computer NetworksInternet RoutingInternet MeasurementSecurity
TC

Thomas C. Schmidt

Professor of Computer Science, HAW Hamburg
Computer NetworksDistributed SystemsInternet TechnologiesInternet Security