Estimating great expectations under autoregressive language models with potentials

2026-10-08

AI Summary

This study addresses the high computational cost of estimating expectations of test functionals in autoregressive language models. To this end, it proposes a variance reduction method based on prefix potential functions. By leveraging the next-token probabilities generated during sampling to decompose the target functional, the approach constructs a low-variance, efficient estimator. Through a potential function mechanism, it significantly reduces the estimation variance of Monte Carlo sampling under equivalent computational budgets. Experimental results demonstrate that the proposed technique achieves substantial variance reduction across diverse estimation tasks, effectively enhancing both estimation efficiency and reliability. Consequently, this work establishes a novel paradigm for the functional evaluation of language models.

Abstract

Many applications of language models hinge not on individual samples but on the expectation of a test functional under the model. Estimating such expectations reliably can be computationally expensive. In this paper, we show how to make estimation more efficient by exploiting the next-token conditional probabilities which are available as a by-product of sampling. We do so through potentials: real-valued functions on prefixes that decompose the test functional additively. We construct an estimator whose variance depends on the chosen potential, and derive conditions under which a potential reduces this variance. We then develop practical potentials for several estimands and applications, and demonstrate substantial variance reductions across several estimands at comparable computational cost.

Problem

• autoregressive language models
• expectation estimation
• variance reduction
• test functional
• computational efficiency

Innovation

• Autoregressive Language Models
• Expectation Estimation
• Potentials
• Variance Reduction
• Conditional Probabilities