Association Restoration Test: Revealing Restorable Shortcuts after Unlearning

2026-07-06

AI Summary

Existing debiasing methods lack an effective evaluation of whether models can still functionally exploit removed label-attribute shortcuts. This work proposes Association Recovery Testing (ART), which assesses debiasing efficacy from a functional perspective by estimating the direction of attribute associations conditioned on class labels, amplifying residual feature signals, and evaluating the modified features using the original classification head to detect recoverable shortcuts. ART reveals residual shortcuts that conventional output-based metrics and representation probes fail to capture, thereby shifting the focus of debiasing approaches from individual classes or isolated concepts toward relational associations. Experiments on Waterbirds, CelebA, SpuCoDogs, and an extended ISIC dataset with timestamp artifacts demonstrate that ART effectively uncovers functional shortcuts persisting after state-of-the-art debiasing interventions.

Abstract

Association unlearning aims to disable learned label-attribute shortcuts while preserving task performance. Existing evaluations mainly measure output-level robustness or probe whether shortcut attributes remain readable in frozen features, but neither test determines whether a retained association remains functionally usable by the original classifier. We propose the Association Restoration Test (ART), a post-hoc diagnostic for functional shortcut restorability. ART estimates class-conditional association directions, amplifies residual components, and evaluates the modified features with the original classifier head. Across Waterbirds, CelebA, SpuCoDogs, and an ISIC timestamp-artifact extension, we show that output metrics, representation probes, and ART characterize distinct aspects of shortcut mitigation. These findings motivate restoration-aware evaluation for unlearning and shortcut-mitigation methods that target learned associations rather than individual classes or concepts.

Problem

• association unlearning
• shortcut mitigation
• functional restorability
• classifier usability
• post-hoc evaluation

Innovation

• Association Restoration Test
• shortcut unlearning
• functional restorability
• post-hoc diagnostic
• representation probing