shortcut detection

Frequent data copyright infringement during large-scale ML model training, coupled with fragmented assumptions, narrow evaluation scopes, and poor cross-method comparability among existing copyright auditing tools, hinders practical deployment. Method: This paper systematically categorizes intrusive (watermark injection) and non-intrusive (fingerprinting-based) auditing paradigms, and—firstly—establishes a unified analytical framework spanning the entire ML pipeline: data collection, preprocessing, training, and inference. Leveraging full-stack ML modeling and controlled cross-method experiments, it characterizes structural trade-offs across assumptions, stage coverage, and real-world robustness. Contribution/Results: It introduces a deployment-oriented evaluation perspective, synthesizes common limitations, and identifies open challenges. The work delivers a taxonomy reference table and a practical implementation guide, providing both theoretical foundations and actionable technical pathways for developing compliant, deployable, and robust data copyright auditing tools.

To address copyright infringement and transparency concerns arising from unauthorized use of third-party data in machine learning model training, this paper proposes the first general-purpose, task-agnostic data usage auditing framework for black-box models. Methodologically, it innovatively integrates arbitrary black-box membership inference techniques with a custom sequential probability ratio test (SPRT), enabling zero assumptions about downstream tasks, strict control over false positive rates (tunable within 0.5%–5%), and cross-model generalization. The framework features a model-agnostic interface, supporting heterogeneous architectures including image classifiers and multimodal large language models. Extensive experiments on ImageNet classifiers and multimodal foundation models demonstrate an average detection accuracy exceeding 92%, with false positive rates consistently meeting user-specified thresholds. This work significantly enhances the quantifiability and reliability of training data provenance auditing.

This work addresses the problem of shortcut learning in binary black-box classification models caused by dataset bias. It proposes a novel post-hoc analysis framework that integrates interventional and observational perspectives, introducing linear mixed-effects models—used here for the first time—to diagnose bias in black-box classifiers. By decomposing the influence of training and test data on model scores, the method moves beyond conventional error-rate metrics to uncover the risk of models relying on spurious correlations. The approach effectively identifies and quantifies the impact of data bias on decision-making in voice anti-spoofing and speaker verification tasks, offering a new pathway toward building reliable and interpretable AI systems.

This work addresses spurious correlations induced by dataset bias. We formally define and quantify “spuriousness” at the dataset level—the first such characterization—enabling systematic diagnosis of misleading statistical dependencies. We propose the Spurious Disentangler framework, which leverages partial information decomposition (PID) to disentangle target-variable information into unique, redundant, and synergistic components, thereby enabling *a priori*, interpretable modeling of dependencies between core and spurious features. The method integrates information-theoretic modeling, nonparametric mutual information estimation, and high-dimensional feature disentanglement, supporting spuriousness assessment on complex data such as images. Experiments across six benchmark datasets demonstrate that our spuriousness metric strongly correlates with worst-group accuracy and other generalization performance measures. The implementation is publicly available.

To address performance degradation, reliance on white-box model information, and high false-positive rates in dataset ownership verification, this paper proposes a black-box, lossless, and zero-false-positive verification framework. Methodologically, it introduces clean-label targeted poisoning to embed a secret key—comprising out-of-distribution samples and random labels—into the training data. Post-training, the model exhibits statistically detectable, significant responses to key samples, without requiring access to internal parameters. Our key contribution is the first non-backdoor-based verification mechanism, integrating statistical hypothesis testing with ViT/ResNet ensembles. On ImageNet-1K, it achieves >99.9% detection confidence and zero accuracy loss. Moreover, it remains robust against common defenses—including pruning, fine-tuning, and input preprocessing—outperforming existing backdoor watermarking approaches significantly.

Existing shortcut learning mitigation methods typically rely on fully annotated data, group-balanced validation sets, or training data that exhaustively covers all attribute–class combinations—conditions rarely met in real-world scenarios. This work proposes a novel approach that requires neither additional annotations nor balanced validation sets. By analyzing internal model representations, the method identifies a small set of samples exhibiting spurious correlations and locates critical neurons responsible for leveraging these misleading attributes, guided by the principle that such features should not inform predictions. Intermediate-layer regularization is then applied to these neurons. Requiring only a few spuriously correlated (false positive) samples, the approach effectively suppresses shortcut learning, significantly enhancing model robustness and preventing models from making correct predictions for incorrect reasons, thereby achieving effective mitigation under more realistic data conditions.

Deep learning models often fail to generalize due to spurious correlations—non-causal statistical associations between features and labels. Existing debiasing methods rely either on manually annotated bias attributes or strong prior assumptions (e.g., bias simplicity), limiting their applicability to real-world data with complex, latent spurious patterns. To address this, we propose a fully data-driven debiasing framework: first, we automatically identify spurious features by measuring clustering dispersion of samples in the feature space; second, we introduce a grouping-based neutralization strategy and a contrastive bias-invariant feature alignment transformation; finally, we jointly optimize the classifier and representation learner in an end-to-end manner. Our approach requires no bias annotations or restrictive assumptions. On standard image and NLP debiasing benchmarks, it improves worst-group accuracy by over 20% relative to Empirical Risk Minimization (ERM), significantly enhancing model robustness and fairness.

This work addresses the risk of spurious correlations between recording artifacts and target labels in speech datasets, which arise from heterogeneous recording conditions and can lead to overestimated model performance—posing significant safety concerns in high-stakes domains such as healthcare. To detect such confounding factors, the study introduces a novel diagnostic method that leverages non-speech segments within audio recordings to predict target labels, exploiting the metadata implicitly embedded in these silent or non-linguistic regions. By analyzing these non-speech intervals, the approach effectively identifies spurious associations present in both training and test data. The authors further release an open-source toolkit implementing this technique, substantially enhancing the reliability of speech model evaluation and the trustworthiness of real-world deployment.

Real-world datasets often exhibit spurious correlations that are not causally related to the target label, leading to poor generalization under distribution shifts. This work proposes a two-stage sample scoring method that operates without group labels by decoupling the learning dynamics of core and spurious features, separately assessing their learning difficulty. Leveraging a memorization-guided mechanism, the method selects an informative subset of samples for debiased training. Remarkably, using only 10% of the original data—carefully curated through this scoring scheme—the approach outperforms state-of-the-art debiasing methods within a standard empirical risk minimization (ERM) framework, achieving superior performance across multiple benchmarks.

This work demonstrates that naturally occurring statistical signals in visual datasets can function as implicit backdoor triggers, enabling manipulation of model predictions without malicious data poisoning. By employing statistical association analysis and conditional control variable methods, the study systematically identifies non-semantic patterns in ImageNet that exhibit strong correlations with specific labels. These patterns are shown to be transferable across diverse model architectures and capable of steering predictions in a controllable manner. The paper introduces the novel concept of “statistical adversarial examples,” revealing that inherent spurious structures in datasets not only serve as sources of bias but also constitute cross-model security vulnerabilities. Notably, such attacks exhibit greater target specificity compared to generic image corruptions.