🤖 AI Summary
Under label shift, existing methods for multiclass classification and anomaly detection suffer from high resampling overhead, unstable coverage for minority classes, and overly conservative prediction sets. To address these issues, we propose MMDCP—a distribution-agnostic framework jointly performing classification and anomaly detection. Its core contributions are threefold: (1) a class-specific Mahalanobis-distance-based scoring function that eliminates the need for data splitting or resampling; (2) the first theoretical characterization of the deviation between empirical and ideal conformal p-values, leading to a cross-class global error metric—SCW-FDR; and (3) rigorous finite-sample guarantees on coverage validity and CW-FDR control under heterogeneous distributions, along with convergence rate analysis of prediction sets. Experiments on synthetic and real-world datasets demonstrate that MMDCP significantly reduces conservativeness, stably controls both coverage probability and SCW-FDR, and improves detection power and computational efficiency.
📝 Abstract
We propose the Modified Mahalanobis Distance Conformal Prediction (MMDCP), a unified framework for multi-class classification and outlier detection under label shift, where the training and test distributions may differ. In such settings, many existing methods construct nonconformity scores based on empirical cumulative or density functions combined with data-splitting strategies. However, these approaches are often computationally expensive due to their heavy reliance on resampling procedures and tend to produce overly conservative prediction sets with unstable coverage, especially in small samples. To address these challenges, MMDCP combines class-specific distance measures with full conformal prediction to construct a score function, thereby producing adaptive prediction sets that effectively capture both inlier and outlier structures. Under mild regularity conditions, we establish convergence rates for the resulting sets and provide the first theoretical characterization of the gap between oracle and empirical conformal $p$-values, which ensures valid coverage and effective control of the class-wise false discovery rate (CW-FDR). We further introduce the Summarized Class-Wise FDR (SCW-FDR), a novel global error metric aggregating false discoveries across classes, and show that it can be effectively controlled within the MMDCP framework. Extensive simulations and two real-data applications support our theoretical findings and demonstrate the advantages of the proposed method.