Modeling Memory-Dependent Reliability of LLMs: A Hidden Markov Model

2026-07-24

AI Summary

Traditional reliability evaluations of large language models typically assume that test samples are independent and identically distributed, thereby overlooking the sequential dependencies arising from memory retention and error propagation in conversational contexts—a limitation that often yields overly optimistic reliability estimates. This work addresses this gap by introducing, for the first time, a hierarchical Bayesian framework grounded in hidden Markov models to explicitly model sequence dependence, capturing how latent interaction states evolve over time and influence output correctness. By relaxing the independence assumption, the proposed approach offers a more realistic representation of real-world interactive scenarios. Empirical evaluations across four experiments on Anthropic’s Claude and OpenAI models demonstrate that neglecting sequential dependencies leads to substantial overestimation of reliability, whereas the new method provides more accurate quantification of both model reliability and its associated uncertainty.

Abstract

Reliability assessment of large language models (LLMs) seeks to estimate the probability that a model produces correct responses under a specified operational profile. Conventional benchmark-based evaluation, often summarized by aggregate accuracy, provides a point estimate of performance but does not characterize the uncertainty associated with reliability claims. Currently, statistical inference methods for LLM reliability assessment are emerging. However, a key assumption underlying these models is that test outcomes can be treated as independent repeated trials. This assumption may be inappropriate in sequential settings, where later responses depend on earlier interactions through retained context, error propagation, or an evolving interaction state. We extend a hierarchical Bayesian framework for LLM reliability assessment by relaxing the assumption of independent task outcomes and introducing a Hidden Markov Model to capture sequential dependence in benchmark-constructed interaction sessions. In this formulation, outcomes are generated from a latent interaction state evolving according to a first-order Markov process, capturing changes in interaction context. Through experiments using Anthropic Claude and OpenAI on four datasets, we demonstrate the potential impact of sequential dependence on reliability assessment. The results suggest that ignoring sequential dependence may lead to overconfident reliability estimates.

Problem

• LLM reliability
• sequential dependence
• memory-dependent responses
• interaction context
• statistical inference

Innovation

• Hidden Markov Model
• sequential dependence
• LLM reliability
• Bayesian framework
• memory-dependent reliability