Open-World Darknet Traffic Recognition Under Leave-One-Service-Out Evaluation

2026-08-04

AI Summary

This study addresses the limitations of existing darknet traffic identification methods, which predominantly rely on closed-world assumptions and struggle to generalize to unknown services in real-world scenarios. To bridge this gap, the authors propose the first open-world darknet traffic classification framework, introducing a novel leave-one-service-out evaluation strategy and an uncertainty-aware rejection mechanism. Their analysis reveals that conventional closed-world evaluations substantially overestimate real-world performance. Experimental results in the I2P environment demonstrate a sharp decline in Macro-F1 scores—from approximately 88% to 46%—for both XGBoost and Random Forest models. Semantic absorption analysis further uncovers significant behavioral overlap between known and unknown services: 88.1% of FreeNet video traffic is misclassified as browsing traffic, and 83.4% of I2P P2P traffic is erroneously labeled as FTP activity.

Abstract

Darknet traffic recognition is critical for cyber threat intelligence, as anonymity networks are often used to conceal malicious activity. However, most existing studies rely on closed-world evaluation, assuming all service categories are known during training and testing, which is unrealistic in real-world environments. This paper presents an open-world darknet traffic classification framework using leave-one-service-out evaluation and uncertainty-aware classification with Random Forest and XGBoost models. Experimental results demonstrate significant performance degradation when transitioning from closed-world to open-world settings, demonstrating that closed-world evaluation substantially overestimates deployment robustness. For example, XGBoost Macro-F1 decreases from 88.8% to 46.1% in the I2P environment, while Random Forest performance drops from 87.4% to 45.7%. Although uncertainty-based rejection slightly improves robustness, strong behavioral similarity between known and unknown services leads to frequent misclassification. Semantic absorption analysis further shows that FreeNet video traffic is classified as browsing traffic with an 88.1% assignment rate, while I2P peer-to-peer traffic is absorbed into FTP-related behavior with an 83.4% assignment rate. The findings demonstrate that behavioral overlap remains a major challenge for reliable open-world darknet traffic classification.

Problem

• open-world
• darknet traffic recognition
• leave-one-service-out
• behavioral overlap
• unknown services

Innovation

• open-world classification
• darknet traffic recognition
• leave-one-service-out evaluation
• uncertainty-aware classification
• semantic absorption