🤖 AI Summary
Bias in Transformer language models is challenging to precisely locate and mitigate. This work proposes ROBIN, a method that performs sensitivity analysis during inference using fairness probes to identify and rank attention heads associated with bias. Rather than naively zeroing out entire heads, ROBIN precisely removes bias directions within the output subspaces of these identified heads. This approach enables white-box fairness debugging at the granularity of individual attention heads. Evaluated across four mainstream Transformer models, ROBIN significantly reduces the WinoBias gap while outperforming naive head-zeroing strategies and better preserving overall language modeling performance.
📝 Abstract
Transformer language models are increasingly used as software components, yet biased outputs remain difficult to localize and repair inside the model. Existing fairness testing and repair methods largely operate at the input-output or retraining level, while recent work suggests that bias-related behavior can concentrate in a small set of attention heads. This paper studies whether attention heads can be localized and repaired through a targeted inference-time intervention. We introduce ROBIN, a white-box head-level fairness debugging method that ranks attention heads using sensitivity to fairness probes and removes a small bias subspace from selected head outputs. In a four-model pilot study, ROBIN reduces the measured WinoBias gap across all models while preserving language-modeling quality better than whole-head zeroing. These preliminary results suggest that head-level bias repair should consider not only which heads are selected, but also how selected heads are modified.