Evaluation Awareness in Language Models Has Limited Effect on Behaviour

2026-05-07

AI Summary

This study investigates whether the expression of evaluation awareness in language models during reasoning induces strategic behavior that compromises the validity of safety and alignment assessments. Through controlled experiments on multiple open-source large reasoning models across benchmarks encompassing safety, alignment, ethics, and political viewpoints, the authors quantify the impact of evaluation awareness using both online (spontaneously emerging) and offline (artificially injected or removed) interventions. Employing chain-of-thought sampling, prefilled interventions, multi-round resampling, and effect size (ω) analysis, they find that injecting evaluation awareness yields negligible effects (ω ≤ 0.06), removing it causes only minor shifts (ω ≤ 0.12), and spontaneously occurring awareness alters answer distributions by at most 3.7 percentage points (ω ≤ 0.31). These results challenge the prevailing assumption that high evaluation awareness necessarily indicates strategic manipulation of alignment evaluations.

Abstract

Large reasoning models (LRMs) sometimes note in their chain of thought (CoT) that they may be under evaluation. Researchers worry that this verbalised evaluation awareness (VEA) causes models to adapt their outputs strategically, optimising for perceived evaluation criteria, which, for instance, can make models appear safer than they actually are. However, whether VEA actually has this effect is largely unknown. We tested this across open-weight LRMs and benchmarks covering safety, alignment, moral reasoning, and political opinion. We tested this both on-policy, sampling multiple CoTs per item and comparing those that spontaneously contained VEA against those that did not, and off-policy, using model prefilling to inject evaluation-aware sentences where missing and remove them where present, with subsequent resampling. VEA has limited effect on model behaviour: injecting VEA into CoTs produces near-zero effects ($ω\leq 0.06$), removing it causes small shifts ($ω\leq 0.12$) and spontaneously occurring VEA shifts answer distributions by at most 3.7 percentage points ($ω\leq 0.31$). Our findings call for caution when interpreting high VEA rates as evidence of strategic behaviour or alignment tampering. Evaluation awareness may pose a smaller safety risk than the current literature assumes.

Problem

• evaluation awareness
• language models
• strategic behavior
• alignment
• safety

Innovation

• evaluation awareness
• chain of thought
• verbalised evaluation awareness
• alignment tampering
• large reasoning models