🤖 AI Summary
This work addresses the problem of shortcut learning in binary black-box classification models caused by dataset bias. It proposes a novel post-hoc analysis framework that integrates interventional and observational perspectives, introducing linear mixed-effects models—used here for the first time—to diagnose bias in black-box classifiers. By decomposing the influence of training and test data on model scores, the method moves beyond conventional error-rate metrics to uncover the risk of models relying on spurious correlations. The approach effectively identifies and quantifies the impact of data bias on decision-making in voice anti-spoofing and speaker verification tasks, offering a new pathway toward building reliable and interpretable AI systems.
📝 Abstract
The widespread adoption of deep-learning models in data-driven applications has drawn attention to thepotential risks associated with biased datasets and models. Neglected or hidden biases within datasets and models can lead to unexpected results. This study addresses the challenges of dataset bias and explores “shortcut learning” or “Clever Hans effect” in binary classifiers. We propose a novel framework for analyzing the black-box classifiers and for examining the impact of both training and test data on classifier scores. Our framework incorporates intervention and observational perspectives, employing a linear mixed-effects model for post-hoc analysis. By evaluating classifier performance beyond error rates, we aim to provide insights into biased datasets and offer a comprehensive understanding of their influence on classifier behavior. The effectiveness of our approach is demonstrated through experiments on audio anti-spoofing and speaker verification tasks using both statistical models and deep neural networks. The insights gained from this study have broader implications for tackling biases in other domains and advancing the field of explainable artificial intelligence. The open-source implementation of the proposed method, along with demonstrations of interventional and observational case analyses.