Diagnosing Pathological Chain-of-Thought in Reasoning Models

2026-02-14

AI Summary

This work addresses critical reliability limitations of chain-of-thought (CoT) reasoning in large language models for AI safety monitoring, identifying three distinct pathological failure modes: post-hoc rationalization, encoded reasoning, and internalized reasoning. The study presents the first systematic characterization and differentiation of these CoT pathologies and introduces a lightweight, task-agnostic, and computationally efficient diagnostic toolkit capable of real-time monitoring during model training. By leveraging behavior-based diagnostic metrics and purpose-built model organisms, the proposed method accurately identifies and distinguishes among the three pathological patterns. This approach offers a practical, low-cost solution to enhance the monitorability and safety of large language models without requiring extensive architectural modifications or computational overhead.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Chain-of-thought (CoT) reasoning is fundamental to modern LLM architectures and represents a critical intervention point for AI safety. However, CoT reasoning may exhibit failure modes that we note as pathologies, which prevent it from being useful for monitoring. Prior work has identified three distinct pathologies: post-hoc rationalization, where models generate plausible explanations backwards from predetermined answers; encoded reasoning, where intermediate steps conceal information within seemingly interpretable text; and internalized reasoning, where models replace explicit reasoning with meaningless filler tokens while computing internally. To better understand and discriminate between these pathologies, we create a set of concrete metrics that are simple to implement, computationally inexpensive, and task-agnostic. To validate our approach, we develop model organisms deliberately trained to exhibit specific CoT pathologies. Our work provides a practical toolkit for assessing CoT pathologies, with direct implications for training-time monitoring.

Problem

• Chain-of-Thought
• reasoning pathologies
• post-hoc rationalization
• encoded reasoning
• internalized reasoning

Innovation

• Chain-of-Thought
• reasoning pathologies
• model organisms
• AI safety
• interpretability