Design-based variance estimation of the H'ajek effect estimator in stratified and clustered experiments

2024-06-15

AI Summary

解决聚类和分层实验中Hájek估计的设计标准误差问题，提出新方差估计器并验证其一致性，结合Neyman估计器适应不同规模分层设计，扩展至协变量处理并保持保守性。

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: State Estimation

Application Category

• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Randomized controlled trials (RCTs) are used to evaluate treatment effects. When individuals are grouped together, clustered RCTs are conducted. Stratification is recommended to reduce imbalance of baseline covariates between treatment and control. In practice, this can lead to comparisons between clusters of very different sizes. As a result, direct adjustment estimators that average differences of means within the strata may be inconsistent. We study differences of inverse probability weighted means of a treatment and a control group -- H'ajek effect estimators -- under two common forms of stratification: small strata that increase in number; or larger strata with growing numbers of clusters in each. Under either scenario, mild conditions give consistency and asymptotic Normality. We propose a variance estimator applicable to designs with any number of strata and strata of any size. We describe a special use of the variance estimator that improves small sample performance of Wald-type confidence intervals. The H'ajek effect estimator lends itself to covariance adjustment, and our variance estimator remains applicable. Simulations and real-world applications in children's nutrition and education confirm favorable operating characteristics, demonstrating advantages of the H'ajek effect estimator beyond its simplicity and ease of use.

Problem

• Estimating causal effects in clustered and stratified experiments
• Addressing inconsistency in ANOVA and other estimators
• Developing design-based standard errors for Hájek estimation

Innovation

• Hájek estimation for clustered, stratified experiments
• New variance estimator for design-based standard error
• Hybrid estimator combining Hájek and Neyman methods