Analyzing Within-Subject Experiments: Identification, Testing, and Sensitivity

📅 2026-08-27
📈 Citations: 0
✨ Influential: 0
📄 PDF
🤖 AI Summary
本文解决了在主体内实验设计中如何准确估计处理效应的问题,通过构建潜在结果框架和敏感性分析方法来评估并改进现有方法。
📝 Abstract
Recent work encourages political scientists to move from post-only toward within-subject designs for improved precision from repeated measurements. We formalize a potential-outcomes framework for two-period within-subject designs that allows for unequal allocation and heterogeneous treatment and carryover effects. We characterize the pooled estimator and evaluate the carryover test used to justify pooling. We find: first, pooling identifies the average treatment effect only when the gap in the average carryover effects is zero across the two treatment sequences. The unit-clustered standard error for the pooled estimator is identical to its design-based counterpart. Second, under mild conditions, the carryover test has strictly less power than the average-treatment-effect test with post-only data. The resulting two-step procedure, which pools only after a nonrejected test, produces confidence intervals that typically undercover. When the gap is zero, undercoverage occurs if and only if pooling is more efficient than post-only analysis, precisely when the within-subject design is worthwhile. When the gap is nonzero, undercoverage is typical unless the gap or sample size is large. Third, we derive a sensitivity analysis and find published conclusions robust to plausible carryover gaps. We therefore endorse within-subject designs but recommend justifying a zero carryover gap substantively and reporting sensitivity to departures.
Problem

Research questions and friction points this paper is trying to address.

within-subject designs
carryover effects
average treatment effect
Innovation

Methods, ideas, or system contributions that make the work stand out.

within-subject designs
carryover effects
sensitivity analysis
pooled estimator
🔎 Similar Papers
2024-06-25arXiv.orgCitations: 0
💼 Related Jobs
No related jobs found.
S
Shiyao Liu
China Center for Economic Research, Institute of South-South Cooperation and Development, National School of Development, Peking University, Beijing, China; Governance Lab, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA
J
Junni L. Zhang
China Center for Economic Research, National School of Development and Center for Statistical Science, Peking University, Beijing, China