🤖 AI Summary
This work addresses the limitations of traditional anomaly detection methods, which rely on heuristic thresholds and lack statistical interpretability and calibration. We propose a conformal prediction–based framework that transforms anomaly scores into statistically valid p-values, enabling rigorous control of the false discovery rate. To facilitate adoption, we introduce nonconform, an open-source Python toolkit that provides the first unified and user-friendly implementation compatible with both scikit-learn and PyOD, incorporating split-conformal inference and efficient calibration strategies robust to distribution shift. Experimental results demonstrate that our approach maintains high detection performance while delivering reliable probabilistic interpretations and strong statistical guarantees.
📝 Abstract
Most anomaly detection systems output scores rather than calibrated decisions, leaving practitioners to choose thresholds heuristically and without clear statistical interpretation. Conformal anomaly detection addresses this limitation by converting anomaly scores into calibrated p-values that are valid under the statistical assumption of data exchangeability, with a growing literature extending this idea beyond that setting. We present 'nonconform', a Python package for applying conformal anomaly detection within existing machine-learning workflows, and use it as the basis for an implementation-grounded introduction to the field. The package integrates with 'scikit-learn', 'pyod', and custom anomaly detectors, and provides a unified interface for calibration, p-value generation, and false discovery rate control. It supports several conformalization strategies, ranging from simple split-conformal calibration to more data-efficient and shift-aware extensions. Through a progression from foundational concepts to advanced conformalization strategies, complemented by code examples, the paper connects the statistical ideas behind conformal anomaly detection to their practical use in 'nonconform'. Empirical results demonstrate that the implemented methods enable statistically principled anomaly detection. Together, the package and exposition aim to make core conformal anomaly detection workflows more accessible and reproducible in experimental and production-oriented settings.