🤖 AI Summary
Existing robustness quantification methods rely on generative models and are often constrained by specific architectures or discrete features, limiting their applicability to general discriminative classifiers. This work proposes a novel robustness metric applicable to any probabilistic discriminative classifier and arbitrary feature types, thereby eliminating dependence on generative models and architectural assumptions and offering, for the first time, an effective means to evaluate robustness in generic discriminative models. Building upon this metric, the authors further design a dynamic classifier selection strategy that effectively distinguishes between reliable and unreliable predictions, significantly improving both selection accuracy and broad applicability.
📝 Abstract
Among the different possible strategies for evaluating the reliability of individual predictions of classifiers, robustness quantification stands out as a method that evaluates how much uncertainty a classifier could cope with before changing its prediction. However, its applicability is more limited than some of its alternatives, since it requires the use of generative models and restricts the analyses either to specific model architectures or discrete features. In this work, we propose a new robustness metric applicable to any probabilistic discriminative classifier and any type of features. We demonstrate that this new metric is capable of distinguishing between reliable and unreliable predictions, and use this observation to develop new strategies for dynamic classifier selection.