🤖 AI Summary
This paper addresses the low statistical power and poor resolution of conventional placebo tests in synthetic control method (SCM) causal inference under small-sample settings—particularly when α = 0.05 and the number of donor units *N* is small. We propose a leave-two-out randomization inference framework that rigorously controls Type I error rates in finite samples while substantially improving test resolution and statistical power, even under stringent significance levels (α < 1/*N*). Unlike permutation or rank-based tests, our framework accommodates non-uniform treatment assignment and integrates formal sensitivity analysis for robust causal inference. Empirical results demonstrate that, under moderate effect sizes, the proposed method achieves lower actual Type I error rates and higher statistical power compared to standard approaches.
📝 Abstract
The synthetic control method is often applied to problems with one treated unit and a small number of control units. A common inferential task in this setting is to test null hypotheses regarding the average treatment effect on the treated. Inference procedures that are justified asymptotically are often unsatisfactory due to (1) small sample sizes that render large-sample approximation fragile and (2) simplification of the estimation procedure that is implemented in practice. An alternative is permutation inference, which is related to a common diagnostic called the placebo test. It has provable Type-I error guarantees in finite samples without simplification of the method, when the treatment is uniformly assigned. Despite this robustness, the placebo test suffers from low resolution since the null distribution is constructed from only $N$ reference estimates, where $N$ is the sample size. This creates a barrier for statistical inference at a common level like $alpha = 0.05$, especially when $N$ is small. We propose a novel leave-two-out procedure that bypasses this issue, while still maintaining the same finite-sample Type-I error guarantee under uniform assignment for a wide range of $N$. Unlike the placebo test whose Type-I error always equals the theoretical upper bound, our procedure often achieves a lower unconditional Type-I error than theory suggests; this enables useful inference in the challenging regime when $alpha<1/N$. Empirically, our procedure achieves a higher power when the effect size is reasonably large and a comparable power otherwise. We generalize our procedure to non-uniform assignments and show how to conduct sensitivity analysis. From a methodological perspective, our procedure can be viewed as a new type of randomization inference different from permutation or rank-based inference, which is particularly effective in small samples.