🤖 AI Summary
This study investigates whether large language models genuinely rely on the reasoning processes demonstrated in chain-of-thought (CoT) prompts when generating answers. To this end, the authors propose two metrics—Causal Mediation Influence (CMI) and Bypass Score—and develop a diagnostic framework integrating hidden-state patching, causal probing, and interpretable signal scoring to quantify the causal dependence of model outputs on CoT reasoning. Experimental results reveal that CMI is near zero across most tasks, indicating that models often bypass the provided CoT and directly produce answers through alternative pathways; only in certain logical reasoning tasks does CMI reach 0.56. Furthermore, while audit-aware prompting improves detection of manipulative signals (increasing risk scores by 5.10), it does not substantially enhance reliance on CoT reasoning, exposing a prevalent phenomenon of superficial compliance with CoT instructions without genuine engagement in the reasoning process.
📝 Abstract
Chain-of-thought (CoT) prompting is widely used as a reasoning aid and is often treated as a transparency mechanism. Yet behavioral gains under CoT do not imply that the model's internal computation causally depends on the emitted reasoning text, i.e., models may produce fluent rationales while routing decision-critical computation through latent pathways. We introduce a causal, layerwise audit of CoT faithfulness based on activation patching. Our key metric, the CoT Mediation Index (CMI), isolates CoT-specific causal influence by comparing performance degradation from patching CoT-token hidden states against matched control patches. Across multiple model families (Phi, Qwen, DialoGPT) and scales, we find that CoT-specific influence is typically depth-localized into narrow"reasoning windows,"and we identify bypass regimes where CMI is near-zero despite plausible CoT text. We further observe that models tuned explicitly for reasoning tend to exhibit stronger and more structured mediation than larger untuned counterparts, while Mixture-of-Experts models show more distributed mediation consistent with routing-based computation. Overall, our results show that CoT faithfulness varies substantially across models and tasks and cannot be inferred from behavior alone, motivating causal, layerwise audits when using CoT as a transparency signal.