Interference Among First-Price Pacing Equilibria: A Bias and Variance Analysis

2024-02-11

arXiv.org

AI Summary

In first-price auction (FPA) markets, budget constraints induce cross-group interference in A/B testing, undermining causal inference validity. Method: We formally characterize the market interference mechanism under the First-Price Pure Equilibrium (FPPE) model; propose a parallel budget-control testing framework based on submarket partitioning to avoid throughput loss from conventional budget splitting; and construct a first-order bias-corrected, unbiased proxy estimator with established asymptotic normality. Our end-to-end causal inference pipeline integrates FPPE modeling, sensitivity analysis, and plug-in estimation. Results: Experiments on large-scale online market data and semi-synthetic benchmarks demonstrate that the framework significantly reduces estimation bias, ensures nominal coverage of confidence intervals, and improves statistical power—particularly under tight budget constraints.

Technology Category

• Game Theory and Economic Paradigms: Auctions and Market-Based Systems
• Reasoning under Uncertainty: Causality
• Machine Learning: Causal Learning

Application Category

• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• Security and Privacy: Large-scale security measurements
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Online A/B testing is widely used in the internet industry to inform decisions on new feature roll-outs. For online marketplaces (such as advertising markets), standard approaches to A/B testing may lead to biased results when buyers operate under a budget constraint, as budget consumption in one arm of the experiment impacts performance of the other arm. To counteract this interference, one can use a budget-split design where the budget constraint operates on a per-arm basis and each arm receives an equal fraction of the budget, leading to ``budget-controlled A/B testing.'' Despite clear advantages of budget-controlled A/B testing, performance degrades when budget are split too small, limiting the overall throughput of such systems. In this paper, we propose a parallel budget-controlled A/B testing design where we use market segmentation to identify submarkets in the larger market, and we run parallel experiments on each submarket. Our contributions are as follows: First, we introduce and demonstrate the effectiveness of the parallel budget-controlled A/B test design with submarkets in a large online marketplace environment. Second, we formally define market interference in first-price auction markets using the first price pacing equilibrium (FPPE) framework. Third, we propose a debiased surrogate that eliminates the first-order bias of FPPE, drawing upon the principles of sensitivity analysis in mathematical programs. Fourth, we derive a plug-in estimator for the surrogate and establish its asymptotic normality. Fifth, we provide an estimation procedure for submarket parallel budget-controlled A/B tests. Finally, we present numerical examples on semi-synthetic data, confirming that the debiasing technique achieves the desired coverage properties.

Problem

• Budget-constrained A/B testing
• Market interference
• First-price auction

Innovation

• Budget-Constrained A/B Testing
• First Price Policy Equilibrium (FPPE) Bias Removal
• Sensitivity Analysis Based Method