Anytime-valid simulation-based hypothesis testing

2026-10-06

AI Summary

This study addresses the challenge of performing effective sequential hypothesis testing on simulated data lacking explicit density functions, a scenario where existing methods often fall short. To overcome this limitation, this work proposes a density-free, efficient solution by integrating martingale theory, e-value statistics, and sequential analysis techniques into a unified e-test martingale framework. The primary contribution lies in enabling anytime-valid sequential testing that simultaneously achieves growth optimality and rigorous error control. Theoretically, the proposed method guarantees strict control of Type I error at arbitrary stopping times, ensures geometric decay of Type II error, and yields asymptotic power approaching unity. Consequently, this framework establishes a reliable anytime-valid paradigm for simulator-driven statistical inference.

Abstract

For a given data distribution $(X_t)_{t \in \mathbb{N}} \sim Q$ i.i.d., we investigate the hypothesis testing problem: $H_0: Q = P_0$ vs. $H_1: Q = P_1$, for two different model probability distributions $P_0$ and $P_1$. In contrast to the standard setting, where analytic densities $p_0$ and $p_1$ are given, here, we consider the density-free setting, where we only have access to i.i.d. simulations $(Z^0_t)_{t \in \mathbb{N}} \sim P_0$ and $(Z^1_t)_{t \in \mathbb{N}} \sim P_1$. For this simulation-based hypothesis testing setting, we construct an e-test martingale, resulting in a sequential test with anytime-valid type-I error guarantees, approximate growth optimality, geometrically decaying type-II error bounds, and asymptotic power one. Most ingredients used in our constructions are variants of well known concepts. The value of this paper lies in the compact presentation of an effective, anytime-valid solution for the density-free simulation-based sequential hypothesis testing case.

Problem

• simulation-based hypothesis testing
• density-free
• sequential testing
• anytime-valid
• e-test martingale

Innovation

• simulation-based hypothesis testing
• e-test martingale
• anytime-valid
• sequential testing
• density-free