🤖 AI Summary
This study addresses the common practice in traditional causal inference of treating diagnostic checks—such as covariate balance, pre-trends tests, and instrumental variable validity—as external to the estimation procedure, which can introduce inferential bias through selective reporting. The paper innovatively endogenizes diagnostic statistics into the estimation process by residualizing (i.e., orthogonalizing) baseline estimators with respect to these diagnostics within a linear adjustment framework. This approach delivers three key advantages: it eliminates bias from selective reporting, reduces variance under correctly specified models, and minimizes worst-case bias under local misspecification. Empirical application to the randomized trial of Kaur et al. (2024) demonstrates that, even when all balance tests are satisfied, the method substantially improves point estimate precision and narrows standard errors—equivalent to a roughly 10% increase in effective sample size.
📝 Abstract
Empirical researchers often use diagnostic checks to assess the plausibility of their modeling assumptions, such as testing for covariate balance in RCTs, pre-trends in event studies, or instrument validity in IV designs. While these checks are traditionally treated as external hurdles to estimation, we argue they should be integrated into the estimation process itself. In particular, we propose residualizing one's baseline estimator against the vector of diagnostic check statistics to remove the component of baseline sampling variation explained by the diagnostic checks. This residualized estimator offers researchers a "free lunch," delivering three properties simultaneously: (i) eliminating inference distortions from check-based selective reporting; (ii) reducing variance without changing the estimand when the baseline model is correctly specified; and (iii) minimizing worst-case bias under bounded local misspecification within the class of linear adjustments. We apply our method to the RCT in Kaur et al. (2024) and find that, even in a setting where all balance checks pass comfortably, residualization increases the magnitude of the baseline point estimate and reduces its standard error, equivalent to approximately a 10% increase in sample size.