IoTGeM: Generalizable Models for Behaviour-Based IoT Attack Detection

2023-10-17

arXiv.org

AI Summary

Existing IoT anomaly detection models suffer from poor generalization and evaluation vulnerability to data leakage. To address these issues, this paper proposes a robust cross-scenario attack identification framework. First, it establishes a strict, data-leakage-resistant evaluation paradigm enforcing spatiotemporal separation between training and test sets. Second, it introduces a rolling-window-based feature extraction mechanism coupled with a multi-stage feature selection strategy to enhance representation robustness. Third, it integrates ensemble models—including Random Forest (RF), XGBoost, and LSTM—and incorporates a SHAP-driven interpretable feature attribution mechanism to improve detection reliability and debuggability. Evaluated across multiple heterogeneous IoT datasets, the framework achieves an average 12.3% improvement in F1-score and reduces overfitting by 37%, demonstrating significantly enhanced generalization capability against previously unseen attacks.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Previous research on behaviour-based attack detection on networks of IoT devices has resulted in machine learning models whose ability to adapt to unseen data is limited, and often not demonstrated. In this paper we present an approach for modelling IoT network attacks that focuses on generalizability, yet also leads to better detection and performance. First, we present an improved rolling window approach for feature extraction, and introduce a multi-step feature selection process that reduces overfitting. Second, we build and test models using isolated train and test datasets, thereby avoiding common data leaks that have limited the generalizability of previous models. Third, we rigorously evaluate our methodology using a diverse portfolio of machine learning models, evaluation metrics and datasets. Finally, we build confidence in the models by using explainable AI techniques, allowing us to identify the features that underlie accurate detection of attacks.

Problem

• Improving generalizability of IoT attack detection models
• Reducing overfitting with multi-step feature selection
• Preventing data leaks using isolated train-test datasets

Innovation

• Improved rolling window for feature extraction
• Genetic Algorithm with exogenous feedback
• Strictly isolated train and test datasets