Decision Theory For Large Scale Outlier Detection Using Aleatoric Uncertainty: With a Note on Bayesian FDR

2025-08-03

AI Summary

This paper addresses anomaly detection in large-scale nonstationary data (e.g., time-varying network traffic). We propose a novel framework integrating Bayesian modeling with decision-theoretic principles. Methodologically, it explicitly models epistemic uncertainty—distinct from conventional approaches focusing solely on aleatoric uncertainty—and introduces a generalized Bayesian false discovery rate (FDR) control mechanism to address calibration challenges in high-dimensional multiple testing. Furthermore, it formulates a decision model based on posterior risk minimization to enhance robustness and interpretability. Empirical evaluation demonstrates that our method significantly outperforms fixed-threshold baselines and state-of-the-art benchmarks on both synthetic time-series data and real-world cybersecurity threat detection tasks. It achieves superior accuracy while maintaining strong adaptability in dynamic, nonstationary environments.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Aleatoric and Epistemic uncertainty have achieved recent attention in the literature as different sources from which uncertainty can emerge in stochastic modeling. Epistemic being intrinsic or model based notions of uncertainty, and aleatoric being the uncertainty inherent in the data. We propose a novel decision theoretic framework for outlier detection in the context of aleatoric uncertainty; in the context of Bayesian modeling. The model incorporates bayesian false discovery rate control for multiplicty adjustment, and a new generalization of Bayesian FDR is introduced. The model is applied to simulations based on temporally fluctuating outlier detection where fixing thresholds often results in poor performance due to nonstationarity, and a case study is outlined on on a novel cybersecurity detection. Cyberthreat signals are highly nonstationary; giving a credible stress test of the model.

Problem

• Develops decision theory for large-scale outlier detection using aleatoric uncertainty
• Introduces Bayesian FDR control for multiplicity adjustment in outlier detection
• Addresses nonstationary cyberthreat signals via novel Bayesian modeling framework

Innovation

• Bayesian framework for aleatoric uncertainty outlier detection
• Generalized Bayesian FDR for multiplicity adjustment
• Nonstationary cybersecurity threat detection application