Latent space bias directions in LLMs capture confidence, not fairness

2026-10-06

AI Summary

This study investigates the inconsistent effectiveness of activation steering for debiasing large language models. Through comparative probing activation analysis and cross-benchmark evaluation, we examine the nature of bias directions in latent space, demonstrating for the first time that linear debiasing vectors actually encode model confidence gradients rather than pure fairness representations. Our findings reveal that activation steering primarily reduces bias metrics by suppressing model confidence; while this improves fairness scores, it simultaneously increases refusal rates. Furthermore, disentangling an independent bias representation from confidence proves intractable. These results expose fundamental limitations inherent in existing steering-based debiasing methods and challenge prevailing assumptions regarding their efficacy.

Abstract

Activation steering has gained popularity as a lightweight inference-time debiasing technique for large language models. However, prior work reports that steering vectors generalise poorly, with unintended effects on model performance and limited transfer to new datasets. Our work analyses what the debiasing direction used for activation steering actually encodes, in order to shed light on its inconsistent performance. We study the linear debiasing direction obtained by contrasting the activations of anti-biased and biased prompts, and evaluate it as a steering intervention across bias and general knowledge benchmarks. We find that this direction is dominated by model confidence, pointing from regions of high to low-probability tokens in activation space rather than encoding a meaningful representation of model bias. Steering along it does reduce measured bias, but this is a consequence of reducing model confidence: on QA benchmarks we find that this steering drives the model to abstain from answering, with a side effect of improving fairness metrics. Our experiments show that model confidence is the dominant separating factor between biased and anti-biased prompts in hidden space, indicating that isolating a linear representation of bias which is disentangled from model confidence is difficult and steering-based debiasing results should be interpreted with care. In short, steering appears to reduce bias, not by correcting the model's underlying preferences, but by making it less confident, even on tasks unrelated to bias.

Problem

• activation steering
• debiasing
• latent space bias
• model confidence
• large language models

Innovation

• Activation Steering
• Latent Space Bias
• Model Confidence
• Debiasing
• Large Language Models