Continuous time asymptotic representations for adaptive experiments

2026-01-02

AI Summary

This work addresses the challenge that adaptive experiments, due to their frequently updated allocation policies, lack a well-defined asymptotic limit, thereby hindering valid statistical inference. To overcome this, the paper proposes a continuous-time asymptotic framework in which the empirical allocation process is approximated by a Gaussian diffusion process with unknown drift. This approximation reduces the effective state space and facilitates optimal decision analysis. The study introduces, for the first time in multi-treatment settings, a formal definition of anytime-valid inference and establishes a continuous-time Gaussian diffusion approximation theory for adaptive experiments. Within this framework, the authors unify the optimal design of estimators, quantification of in-sample regret, and construction of anytime-valid inference procedures.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

This article develops a continuous-time asymptotic framework for analyzing adaptive experiments -- settings in which data collection and treatment assignment evolve dynamically in response to incoming information. A key challenge in analyzing fully adaptive experiments, where the assignment policy is updated after each observation, is that the sequence of policy rules often lack a well-defined asymptotic limit. To address this, we focus instead on the empirical allocation process, which captures the fraction of observations assigned to each treatment over time. We show that, under general conditions, any adaptive experiment and its associated empirical allocation process can be approximated by a limit experiment defined by Gaussian diffusions with unknown drifts and a corresponding continuous-time allocation process. This limit representation facilitates the analysis of optimal decision rules by reducing the dimensionality of the state-space and leveraging the tractability of Gaussian diffusions. We apply the framework to derive optimal estimators, analyze in-sample regret for adaptive experiments, and construct e-processes for anytime-valid inference. Notably, we introduce the first definition of any-time and any-experiment valid inference for multi-treatment settings.

Problem

• adaptive experiments
• asymptotic analysis
• treatment assignment
• continuous-time framework
• policy rules

Innovation

• continuous-time asymptotics
• adaptive experiments
• Gaussian diffusions
• empirical allocation process
• anytime-valid inference