🤖 AI Summary
This work addresses the prevalent yet often undetectable issue of logical inconsistency between reasoning and final answers in chain-of-thought (CoT) outputs generated by current AI systems during safety evaluations. The study is the first to formally distinguish between reasoning consistency and faithfulness, introducing a taxonomy encompassing six distinct types of inconsistency. To enable post-hoc detection without modifying model generation, the authors propose InspectScout—a reusable scanning method grounded in formal definitions, supported by a human-annotated benchmark, and implemented via an automated detection algorithm integrated into the inspect_evals framework. Experiments demonstrate that reasoning inconsistencies are widespread across four mainstream models and three safety-related tasks, and can be reliably identified; moreover, the patterns of such inconsistencies exhibit systematic variation across models.
📝 Abstract
Prior work has shown that chain-of-thought (CoT) reasoning is often unfaithful: a model's stated reasoning does not reliably reflect the process that produced its output. Detecting unfaithfulness, though, requires controlled experimental interventions, which cannot be applied to evaluation transcripts after the fact. We turn instead to a more tractable question that has received less attention: whether the stated reasoning is logically consistent with the answer it accompanies. Unlike faithfulness, consistency can be assessed from a transcript alone, with no intervention. We introduce reasoning consistency scanning, a reusable method for detecting this property in AI safety evaluation transcripts. Our contributions are fourfold. First, we formalize reasoning consistency as distinct from faithfulness and define a six-subtype taxonomy of inconsistency. Second, we build a validated benchmark of 60 transcripts, manually adapted from InstrumentalEval outputs. Third, we implement a working scanner for InspectScout, the first to target this property in safety evaluation transcripts. Fourth, we report results across four generator models and three evaluations from inspect_evals, showing that reasoning inconsistency is present, detectable, and varies systematically across both models and task types.