🤖 AI Summary
Traditional fixed-sample hypothesis tests lack temporal flexibility—they cannot be terminated early or extended adaptively without inflating Type I error.
Method: We propose an anytime-valid sequential testing framework that preserves statistical power. We rigorously prove that any fixed-sample test can be equivalently transformed into a sequential counterpart with identical power. This is achieved via p-value reconstruction and reinterpretation of significance levels, ensuring that the test can be stopped or continued at any time while strictly controlling the overall Type I error rate.
Contributions/Results: We derive explicit anytime-valid versions of the z-test and t-test, which coincide exactly with their classical fixed-sample counterparts after N observations. We further show that the log-optimal sequential z-test corresponds to rejecting the null at the minimal future significance level required by the standard z-test. Our framework unifies fixed-sample and sequential paradigms, enabling reliable inference under dynamic, real-time data collection.
📝 Abstract
Anytime valid sequential tests permit us to stop and continue testing based on the current data, without invalidating the inference. Given a maximum number of observations $N$, one may believe this must come at the cost of power when compared to a conventional test that waits until all $N$ observations have arrived. Our first contribution is to show that this is false: for any valid test based on $N$ observations, we derive an anytime valid sequential test that matches it after $N$ observations. Our second contribution is that the outcome of a continuously-interpreted test can be used as a significance level in subsequent testing, leading to an overall procedure that is valid at the original significance level. This shows anytime validity and optional continuation are readily available in traditional testing, without requiring explicit use of e-values. We illustrate this by deriving the anytime valid sequentialized $z$-test and $t$-test, which at time $N$ coincide with the traditional $z$-test and $t$-test. Lastly, we show the popular log-optimal sequential $z$-test can be interpreted as desiring a rejection by the traditional $z$-test at some tiny significance level in the distant future.