🤖 AI Summary
This study investigates the impact of label noise on model abstention behavior in out-of-distribution (OOD) classification. We propose BCOPS, a conformal prediction-based algorithm that— for the first time under label noise—constructs prediction sets with statistically guaranteed coverage, and systematically evaluates its abstention rate and robustness on unseen classes during training. Experiments on synthetic data and real-world benchmarks (e.g., CIFAR-10/CIFAR-100) reveal that even low-level label noise (≤5%) substantially increases OOD abstention rates, undermining model reliability. Our key contributions are threefold: (i) uncovering an implicit interference mechanism by which label noise distorts OOD abstention; (ii) empirically characterizing the degradation pattern of BCOPS’s coverage fidelity under noise; and (iii) establishing a novel evaluation paradigm for noise-robust trustworthy anomaly detection.
📝 Abstract
This study investigates the impact of adding noise to the training set classes in classification tasks using the BCOPS algorithm (Balanced and Conformal Optimized Prediction Sets), proposed by Guan&Tibshirani (2022). The BCOPS algorithm is an application of conformal prediction combined with a machine learning method to construct prediction sets such that the probability of the true class being included in the prediction set for a test observation meets a specified coverage guarantee. An observation is considered an outlier if its true class is not present in the training set. The study employs both synthetic and real datasets and conducts experiments to evaluate the prediction abstention rate for outlier observations and the model's robustness in this previously untested scenario. The results indicate that the addition of noise, even in small amounts, can have a significant effect on model performance.