Epistemic Throughput: Fundamental Limits of Attention-Constrained Inference

2026-02-09

AI Summary

This work proposes the Attention-Constrained Inference (ACI) framework for scenarios where AI systems can inexpensively generate a large pool of candidates but afford only sparse, costly verification. The framework decomposes inference into a cheap screening phase followed by an expensive validation phase. Through an information-theoretic analysis under Bayesian log loss, the authors define “cognitive throughput” as the maximal achievable reduction in posterior uncertainty per unit window and establish its theoretical upper bound. The central contribution is the “JaKoB” scaling law, which shows that cognitive throughput comprises a linear baseline term plus a √(JKB) information-leverage term, revealing that expanding the screening scale nonlinearly amplifies the effectiveness of scarce validations. Moreover, significant gains under sparse validation occur only when the scoring distribution exhibits heavy tails.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search

Application Category

• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Recent generative and tool-using AI systems can surface a large volume of candidates at low marginal cost, yet only a small fraction can be checked carefully. This creates a decoder-side bottleneck: downstream decision-makers must form reliable posteriors from many public records under scarce attention. We formalize this regime via Attention-Constrained Inference (ACI), in which a cheap screening stage processes $K$ records and an expensive verification stage can follow up on at most $B$ of them. Under Bayes log-loss, we study the maximum achievable reduction in posterior uncertainty per window, which we call \emph{epistemic throughput}. Our main result is a ``JaKoB''scaling law showing that epistemic throughput has a baseline term that grows linearly with verification and prevalence, and an additional \emph{information-leverage} term that scales as $\sqrt{JKB}$, where $J$ summarizes screening quality. Thus, expanding cheap screening can nonlinearly amplify scarce verification, even when informative records are rare. We further show that this scaling is tight in a weak-screening limit, and that in the sparse-verification regime ($B \ll K$), substantial leverage requires heavy-tailed score distributions; for light-tailed scores the amplification is only logarithmic.

Problem

• Attention-Constrained Inference
• epistemic throughput
• posterior uncertainty
• verification bottleneck
• information leverage

Innovation

• epistemic throughput
• attention-constrained inference
• information leverage
• scaling law
• Bayesian inference