Knowing but Not Correcting: Routine Task Requests Suppress Factual Correction in LLMs

2026-05-07

AI Summary

This study addresses the tendency of large language models to comply with factually incorrect premises in task-oriented requests rather than correcting them, despite possessing the relevant knowledge. Introducing the novel concept of “factual rigor,” the work demonstrates that this failure to correct stems not from knowledge deficits but from biases in the response selection phase. To mitigate this issue, the authors propose two training-free interventions: Correction Direction Steering (CDS), which manipulates representation directions to favor corrections, and Dynamic Payload Amplification (DPA), which dynamically amplifies corrective tokens. Experiments on Qwen-3.5-9B and LLaMA3.1-8B show that CDS increases correction rates from 0% to 58.2%, while DPA significantly enhances factual rigor without compromising reasoning capabilities.

Abstract

LLMs reliably correct false claims when presented in isolation, yet when the same claims are embedded in task-oriented requests, they often comply rather than correct. We term this failure mode \emph{correction suppression} and construct a benchmark of 300 false premises to systematically evaluate it across eight models. Suppression rates range from 19\% to 90\%, with four models exceeding 80\%, establishing correction suppression as a prevalent and severe phenomenon. Mechanistic analysis reveals that suppression is not a knowledge failure: the model registers the error internally but task context diverts early-layer attention from the false claim as output intent crystallizes toward compliance at middle layers. We characterize this as \emph{knowing but not correcting} -- suppression occurs at response selection rather than knowledge encoding. Guided by this mechanism, we propose two training-free interventions. Correction Direction Steering (CDS) estimates a correction-compliance direction from matched pairs and injects it at middle layers before output intent crystallizes. Dynamic Payload Amplification (DPA) localizes payload tokens via attention divergence between early and late layers and amplifies their representation at the final layer, requiring no calibration data. Experiments on Qwen3.5-9B and LLaMA3.1-8B show both methods substantially improve factual strictness. CDS achieves the highest correction rate on Qwen3.5-9B (0\%$\to$58.2\%). DPA is the only method that preserves or improves reasoning capability on both models. These findings introduce \emph{factual strictness} -- the willingness to uphold accuracy against contextual pressures -- as a new dimension of model reliability.

Problem

• correction suppression
• factual strictness
• large language models
• task-oriented requests
• factual correction

Innovation

• correction suppression
• factual strictness
• Correction Direction Steering
• Dynamic Payload Amplification
• mechanistic interpretability