AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Games

2026-08-06

AI Summary

This work addresses the inefficiency of traditional agent evaluation in imperfect-information games, where stochasticity necessitates a large number of matches, and fixed-budget or naive stopping rules often waste resources or compromise statistical validity. The paper introduces the first evaluation framework that enables anytime stopping by integrating the AIVAT variance reduction technique with confidence sequences—specifically Asymptotic CS and Empirical-Bernstein CS. The method leverages historical matches to learn correction terms without data leakage and establishes structural bounds on corrected returns, decoupling asymptotic screening from finite-sample exact verification. In HUNL Texas Hold’em experiments, the approach reduces the median number of matches by 74× compared to baselines under Asymptotic CS, while achieving a median stopping time only 1.37× higher under Empirical-Bernstein CS, substantially improving evaluation efficiency without sacrificing rigorous statistical guarantees.

Abstract

Deciding which of two agents is stronger means playing games until skill outweighs luck, and every game costs money, model inference, or expert time. Since the number of games needed is unknown, fixed-budget evaluations either keep paying after the result is settled or stop before the agents can be told apart, while naive optional stopping with an ordinary confidence interval invalidates the stated level. We make such an evaluation stop as soon as its evidence suffices, with the guarantee intact. The Action-Informed Value Assessment Tool (AIVAT) reduces variance in imperfect-information games through conditional mean-zero corrections, by a median $54\times$ across 15 LLM agent configurations spanning 71,439 paired Heads-Up No-Limit Hold'em (HUNL) hands, but does not say when to stop. We combine AIVAT with continuously monitored Confidence Sequences (CSs) into anytime-valid AIVAT (AV-AIVAT), whose online value model learns only from past games so that no game scores its own correction. At the nominal 95\% level and a target precision of $\pm1$ Big Blind, raw outcomes need a median $74\times$ as many hands as AIVAT-corrected outcomes to stop under the Asymptotic CS (AsympCS). Exact finite-sample certification uses the Empirical-Bernstein CS (EB-CS), which needs an independently justified bound on corrected payoffs. We establish such a bound structurally for Leduc hold'em and characterize a width floor set by the CS's bet cap and that bound, which governs how much of a variance gain becomes earlier stopping; the descriptive HUNL EB-CS runs show a median $1.37\times$ stopping-time ratio. AV-AIVAT turns variance reduction into efficient, auditable early stopping while separating asymptotic screening from exact certification, so an evaluation can stop the moment its evidence suffices and hand a third party everything needed to recheck the verdict at that very stopping time.

Problem

• agent evaluation
• imperfect-information games
• anytime-valid stopping
• variance reduction
• confidence sequences

Innovation

• Anytime-Valid Inference
• Variance Reduction
• Confidence Sequences
• Imperfect-Information Games
• Early Stopping