🤖 AI Summary
This work addresses the challenge of dynamic reasoning failures in large language models—such as “mid-reasoning distraction”—which often lead to incorrect outputs but remain undetected by conventional evaluations that focus solely on final answers. To capture such process-level breakdowns, the authors propose a training-free, model-agnostic diagnostic method that leverages token-level log probabilities available during inference. By combining Jensen-Shannon divergence and entropy, they construct a dynamic instability metric that distinguishes between “constructive” and “destructive” instability, revealing how the timing of instability critically influences the model’s capacity for self-correction. Experiments on GSM8K and HotpotQA demonstrate that peak instability intensity strongly predicts reasoning errors (with AUC significantly above random chance) and exhibits a monotonic decline in accuracy as model scale increases, confirming the method’s effectiveness and scalability.
📝 Abstract
Reasoning failures in large language models (LLMs) are typically measured only at the end of a generation, yet many failures manifest as a process-level breakdown: the model"loses the thread"mid-reasoning. We study whether such breakdowns are detectable from inference-time observables available in standard APIs (token log probabilities), without any training or fine-tuning. We define a simple instability signal that combines consecutive-step distributional shift (JSD) and uncertainty (entropy), summarize each trace by its peak instability strength, and show that this signal reliably predicts failure. Across GSM8K and HotpotQA, instability strength predicts wrong answers with above-chance AUC and yields monotonic bucket-level accuracy decline at scale across model sizes. Crucially, we show that instability is not uniformly harmful: early instability can reflect subsequent stabilization and a correct final answer (\emph{corrective instability}), whereas late instability is more often followed by failure (\emph{destructive instability}), even at comparable peak magnitudes, indicating that recoverability depends not only on how strongly the distribution changes but also on when such changes occur relative to the remaining decoding horizon. The method is model-agnostic, training-free, and reproducible, and is presented as a diagnostic lens rather than a corrective or control mechanism.