🤖 AI Summary
This work addresses the challenge of evaluating long-term causal effects of online advertising mechanism changes—such as reserve price adjustments—which not only affect immediate revenue but also alter user behavior, advertiser bidding, and retention dynamics, thereby violating the i.i.d. assumption underlying conventional causal inference. To overcome this limitation, the study introduces, for the first time, a stopped random walk model combined with a budget-split experimental design. Leveraging Anscombe’s theorem, Wald-type equations, and the central limit theorem, it constructs asymptotically unbiased confidence intervals for long-term treatment effects. This approach explicitly accounts for the dynamic interplay among user retention, advertiser budgets, and mechanism parameters, providing a robust causal evaluation framework that transcends the i.i.d. constraint and enables reliable assessment of long-term impacts from advertising policy changes.
📝 Abstract
We consider a causal inference problem frequently encountered in online advertising systems, where a publisher (e.g., Instagram, TikTok) interacts repeatedly with human users and advertisers by sporadically displaying to each user an advertisement selected through an auction. Each treatment corresponds to a parameter value of the advertising mechanism (e.g., auction reserve-price), and we want to estimate through experiments the corresponding long-term treatment effect (e.g., annual advertising revenue). In our setting, the treatment affects not only the instantaneous revenue from showing an ad, but also changes each user's interaction-trajectory, and each advertiser's bidding policy -- as the latter is constrained by a finite budget. In particular, each a treatment may even affect the size of the population, since users interact longer with a tolerable advertising mechanism. We drop the classical i.i.d. assumption and model the experiment measurements (e.g., advertising revenue) as a stopped random walk, and use a budget-splitting experimental design, the Anscombe Theorem, a Wald-like equation, and a Central Limit Theorem to construct confidence intervals for the long-term treatment effect.