Perturbation Sensitivity at Convergence: A Simple Signal for Identifying Spuriously Correlated Samples

2026-08-05

AI Summary

When models are trained on data containing spurious correlations, they often achieve high overall accuracy but exhibit significantly degraded performance on subpopulations where such correlations do not hold. Existing debiasing methods typically rely on group-labeled validation sets to determine when to intervene during training. This work proposes a novel approach that operates without access to group labels: after full model convergence, it identifies vulnerable samples—those influenced by spurious correlations—by measuring prediction flip frequency under fixed input perturbations across two forward passes. Samples affected by spurious correlations yield stable predictions, whereas others are more susceptible to flips. By eliminating the need for early intervention or group annotations, the method improves worst-group accuracy on the Waterbirds dataset from 57.3% to 80.8%, approaching the performance of oracle methods that use true group labels (85.8%).

Abstract

Models trained by empirical risk minimization on data containing spurious correlations achieve high average accuracy while failing on subpopulations where the correlation does not hold. Existing methods for identifying the affected samples without group annotations rely on signals from early training, which requires locating the epoch at which to intervene, a hyperparameter typically selected using group-labeled validation data. We show that a usable signal is available after convergence, when loss no longer distinguishes the two populations. Samples consistent with the spurious correlation are classified by a shared rule, while the remaining samples are fit through configurations specific to individual inputs and are correspondingly more fragile. Applying a fixed perturbation to a converged model's inputs flips the predictions of the latter far more often than the former. The resulting procedure requires two forward passes per training sample, no group annotations at any stage, and no early-stopping epoch. Using the detected samples to rebalance training raises worst-group accuracy on Waterbirds from 57.3% to 80.8%, against 85.8% with ground-truth group labels.

Problem

• spurious correlations
• worst-group accuracy
• group annotations
• model convergence
• sample identification

Innovation

• spurious correlations
• perturbation sensitivity
• converged models
• group-agnostic detection
• worst-group accuracy