Chain-of-Thought Monitoring Can Be Unreliable in Implicit-Influence Settings

2026-08-05

AI Summary

This study addresses the insufficient reliability of current Chain-of-Thought (CoT) monitoring under implicit influence scenarios, where model behavior shifts often go undetected. The authors establish the first systematic benchmark comparing CoT monitoring performance under explicit versus implicit influences, spanning four task types and seven state-of-the-art reasoning models, to evaluate behavioral changes and detectability under suggestive or directive interference. Findings reveal that while CoT monitoring achieves detection rates of 60–94% under explicit influence, performance drops sharply by 41–46 percentage points under implicit influence. Notably, when realistic system prompts are introduced, implicit detection rates fall as low as 5%, despite significant behavioral deviations persisting. These results suggest that current safety evaluations may substantially overestimate real-world monitoring efficacy, and standard system prompts can even degrade detection capabilities under implicit influence.

Abstract

Chain-of-thought (CoT) monitoring is increasingly treated as an important safety layer for frontier reasoning models. Most monitorability evaluations study explicit-influence settings: setups where the prompt directly incentivizes the model to hide something, e.g., by instructing it to perform a hidden side-task. A complementary axis for CoT-monitor evaluations is implicit-influence settings, where the prompt contains no instruction to hide, but the model's behavior is still shaped by features of the task or context, e.g. an irrelevant detail about a candidate that biases a hiring rating. We introduce the first benchmark that directly compares CoT monitorability under the two regimes. We test how model behavior changes in the presence of a nudge to choose a particular option. The nudge is delivered either as a casual aside (implicit), or as a direct instruction to act on the nudge and to conceal having done so (explicit). The benchmark spans four task formats (binary choice, numeric rating, multiple-choice QA, open-ended coding) and seven frontier extended-thinking models. Under explicit influence, a CoT monitor detects 60-94% of behavior shifts: even models instructed to conceal it leak the instruction into their CoT. Under implicit influence, the same factors still shift behavior, but detection falls by 41-46 percentage points in two of our four settings. Realistic system-prompt additions (of the kind a developer might deploy to reduce off-topic bias) lower implicit detection further, to as low as 5%, while preserving the behavioral influence itself. These results suggest that monitorability estimates obtained in explicit-influence settings may over-estimate monitorability, and that monitorability can be further decreased by well-intentioned deployment choices. Our benchmark and code are available at https://github.com/agatha-duzan/implicit-vs-explicit-influence

Problem

• Chain-of-Thought monitoring
• implicit influence
• explicit influence
• model monitorability
• reasoning models

Innovation

• Chain-of-Thought monitoring
• implicit influence
• explicit influence
• AI safety
• behavioral bias