detection performance analysis

This paper addresses the challenge of analytically constructing the optimal ROC curve in binary hypothesis testing when prior distributions are unknown or intractable. We propose the first maximum likelihood estimator for the ROC curve based on observed likelihood ratio samples (MLE-ROC). Unlike conventional approaches, MLE-ROC operates directly on likelihood ratio samples without requiring knowledge of the underlying data distributions and exhibits strong convergence under the Lévy metric. We establish theoretical consistency of its AUC estimator and demonstrate—via simulations—that it significantly outperforms the empirical ROC estimator, especially in small-sample and highly imbalanced settings with sparse negative instances, reducing estimation error by over 40%. The key contribution is the first formal parameterization of the ROC curve in the likelihood ratio domain within a maximum likelihood estimation framework, accompanied by rigorous asymptotic statistical guarantees.

Conventional stochastic geometry methods focus solely on SINR statistics and fail to characterize the statistical variability of radar ROC performance across network realizations. Method: This work establishes, for the first time, an analytical framework for the conditional ROC distribution under Poisson-distributed interference and clutter fields. It derives closed-form expressions for the mean and variance of false-alarm and detection probabilities; employs the Cantelli inequality to obtain tight upper bounds; and approximates the noise-plus-interference power ratio via a Beta-distributed meta-distribution to statistically characterize the conditional ROC. Contribution/Results: Leveraging the higher-order Campbell–Mecke theorem and stochastic geometric analysis, we formulate a quantile-guaranteed robust ROC design paradigm. This provides a rigorous theoretical foundation and novel design principles for detection threshold selection and signal processing in high-reliability radar systems.

The area under the ROC curve (AUC) is commonly interpreted as the probability that a classifier ranks a randomly chosen positive instance higher than a negative one; however, this interpretation relies on specific assumptions whose violation can introduce bias that has not been rigorously characterized. This work systematically reviews the relevant literature and, drawing on probabilistic and statistical methods, provides the first rigorous proof of the conditions under which this probabilistic interpretation holds. Furthermore, when these assumptions are violated, the study derives a computable upper bound on the resulting deviation. By establishing a solid theoretical foundation for the probabilistic interpretation of AUC and offering explicit error bounds, this research significantly enhances the reliability and practical applicability of ROC analysis.

This work investigates the geometric foundations of ROC and PR curves in binary classification, aiming to unify the understanding of curve morphology and classifier behavior through a geometric lens. Methodologically, it introduces the composite function (G = F_p circ F_n^{-1}) as a core modeling framework—where (F_p) and (F_n) denote the CDFs of positive and negative class score distributions—and rigorously establishes a geometric mapping between ROC/PR curve shapes and the underlying distributional geometry. It reveals that (G) quantifies inter-class leakage and admits interpretation via KL divergence. Furthermore, it derives geometric criteria for classifier dominance and interpretability grounded in differential geometry, statistical inference, and CDF transformation theory. The contributions include: (i) a principled, geometrically interpretable framework for threshold selection; (ii) robust, distribution-agnostic tools for classifier comparison; and (iii) enhanced reliability and adaptability in cost-sensitive deployment—particularly under class imbalance and distributional overlap.

This study addresses sequential multi-stream detection under the constraint that only one data stream can be observed at each time step, with the goal of simultaneously controlling global false alarm and missed detection probabilities while minimizing detection delay. To this end, the work introduces a novel optimality criterion based on the expected order statistics of detection times and proposes an active sampling strategy—dubbed “follow-the-leader”—that integrates exploration and exploitation mechanisms. Theoretical analysis demonstrates that the proposed strategy achieves asymptotic optimality for all such criteria as error probabilities vanish. Numerical experiments further confirm its superior finite-sample performance compared to existing methods and show that it closely approaches the performance of an ideal oracle policy that has full knowledge of the anomalous streams.

This study addresses the challenge of effectively integrating radiologists’ assessments with AI predictions in mammographic screening to optimize rule-out and rule-in diagnostic strategies. It introduces, for the first time, a unified joint ROC theoretical framework tailored to both clinical scenarios. By modeling the dependence between physician and AI diagnostic outputs using bivariate copulas, the work theoretically derives—and empirically validates—the impact of their correlation on AUC performance: higher correlation improves rule-out efficacy in diseased populations, whereas lower correlation is preferable in non-diseased populations; conversely, for rule-in tasks, the opposite pattern holds. This framework provides a rigorous theoretical foundation and practical guidance for designing collaborative diagnostic systems that strategically leverage human–AI synergy.

This study addresses the critical issue that existing selective prediction methods in signal domains—such as anomalous sound detection and AI-generated image forensics—often yield a false sense of security due to the use of uncalibrated thresholds, resulting in actual error rates that substantially exceed users’ prescribed risk budgets. The work presents the first systematic audit of four distribution-free calibration rules (NAIVE, Hoeffding, Clopper–Pearson, and Betting) regarding their risk control performance on both real and synthetic data. Findings reveal that NAIVE exceeds the risk budget in 49–73% of experiments; Clopper–Pearson and Betting achieve zero violations under exchangeability but suffer 9–30% violation rates when deployed in grouped settings where exchangeability fails. Group-wise thresholding restores valid risk control at the cost of reduced coverage. The study underscores the pivotal role of tight confidence bounds for effective coverage and identifies uncalibrated thresholds as the root cause of risk miscontrol.

This work addresses the confounding of physical process modeling capability and alarm thresholding effects in existing evaluations of cyber-physical system (CPS) anomaly detectors, which obscures the attribution of performance differences. To resolve this, the authors decouple detection into two stages—residual generation and threshold-based alarming—and propose a normalized residual energy–based evaluation metric. This metric independently quantifies a model’s ability to represent the underlying physical process without relying on specific decision rules or hyperparameter tuning. Furthermore, it connects to KL divergence to measure attack separability, training–testing stability, and model compactness. Evaluations across five detector families on the SWaT, WADI, and HAI benchmarks reveal that performance rankings are highly scenario-dependent and precisely identify failure causes—such as inadequate representation capacity, suboptimal thresholds, or weak physical manifestations of attacks.

This study addresses the overreliance on the Area Under the ROC Curve (AUC) in software defect prediction research, which can lead to biased model evaluation as AUC fails to capture a model’s discriminative performance across all classification thresholds. To overcome this limitation, the authors propose an augmented ROC curve and alternative visualization techniques that explicitly model the true positive rate and false positive rate as functions of the decision threshold, annotating corresponding threshold points directly on the ROC curve. Their analysis demonstrates that a high AUC does not guarantee superior performance over random guessing at every threshold, thereby exposing critical shortcomings of conventional evaluation practices. The work underscores the necessity of incorporating multi-threshold classification performance into a more comprehensive and nuanced assessment framework.