Reliability Scales Inversely: Bigger Models Compound Mistakes Faster via a Hidden Auto-Regressive Risk Regime

2026-06-30

AI Summary

This work investigates the reliability degradation of large language models (LLMs) at scale, revealing that while initial accuracy improves with model size, low-probability tokens trigger a snowballing error effect that drastically undermines trustworthiness. The study quantifies, for the first time, the asymmetric scaling between knowledge acquisition and degradation, uncovering a dominant, self-sustaining error propagation mechanism that models struggle to detect. By leveraging a stronger in-family model as an oracle and employing second-moment decomposition of log-probability gaps (δ), KL divergence, variance analysis, and semantic entropy as self-supervised uncertainty metrics, the authors demonstrate that scaling amplifies knowledge degradation by 39× and extends hallucination persistence on high-risk branches by 23×. Crucially, fixing KL variance reduces validation hallucinations by 35%–74%, whereas existing self-monitoring methods suffer a ~30% drop in detection efficacy on such high-risk paths.

Abstract

As language models scale, answers start truer but degrade faster: scaling buys capability but erodes reliability. The knowledge-gap account - more data, retrieval, or scale - misses an auto-regressive risk residual that scale sharpens: the model commits to a low-probability token, conditions on it as established, and snowballs. We track this through per-position disagreement $\delta = \log p_M - \log p_O$ against a stronger same-family oracle, whose second moment splits exactly into bias$^2$ $\mathrm{KL}(p_M \,\|\, p_O)^2$ and risk $\mathrm{Var}[\delta]$. We present four findings: (i) under scaling, the knowledge gap falls $\approx$$6\times$ while knowledge degradation grows $11$-$39\times$; (ii) at a fabrication, felt uncertainty $H(p_M)$ relaxes quickly while oracle-referenced risk persists up to $17\times$ longer, leaving a confident-but-precarious risk regime that bridges consecutive fabrications ($+69\%$ at $14$B); (iii) this regime is causal - an on-policy, fixed-$\mathrm{KL}$ variance contraction cuts web-verified hallucination by $35$-$74\%$ across three model families; and, (iv) it structurally evades self-monitoring, with $p_M$-only detectors (e.g. semantic entropy) firing $\approx$$30\%$ less ($p<10^{-16}$) on the risky branch holding nearly $4\times$ more fabrications. Bigger models snowball mistakes faster, through a failure mode that is dominant, self-perpetuating, causal and invisible to the model itself.

Problem

• reliability
• scaling
• hallucination
• autoregressive risk
• language models

Innovation

• autoregressive risk
• knowledge degradation
• oracle-referenced variance
• hallucination propagation
• self-monitoring failure