Omitted-Variable Sensitivity Analysis for Generalizing Randomized Trials

2026-03-29

AI Summary

This study addresses the external validity bias that arises when generalizing results from randomized controlled trials to a target population due to distributional differences in unobserved effect modifiers. Building on omitted variable bias theory, the authors propose a novel sensitivity analysis framework that decomposes external validity bias into the product of effect modification strength and covariate distributional imbalance. The framework introduces a dimensionless sensitivity parameter based on partial R², enabling closed-form bias bounds and facilitating benchmarking against observed covariates. Simulation studies demonstrate that the proposed bounds remain conservative and achieve nominal coverage even under model misspecification, offering improved interpretability and practical utility compared to existing approaches.

Abstract

Randomized controlled trials (RCTs) yield internally valid causal effect estimates, but generalizing these results to target populations with different characteristics requires an untestable selection ignorability assumption: conditional on observed covariates, trial participation must be independent of potential outcomes. This assumption fails when unobserved effect modifiers are distributed differently between trial and target populations. We develop a sensitivity analysis framework for trial generalization grounded in omitted variable bias (OVB). Our key theoretical contribution is an exact decomposition showing that external-validity bias equals moderation strength $\times$ moderator imbalance: (i) how strongly an unobserved variable shifts the treatment effect, times (ii) how differently that variable is distributed across populations after covariate adjustment. We introduce scale-free sensitivity parameters based on partial $R^2$ values, enabling closed-form bounds and benchmarking against observed covariates -- practitioners can assess whether conclusions would change if an unobserved moderator were "as strong as" a particular observed variable. Simulations demonstrate that our bounds achieve nominal coverage and remain conservative under model misspecification, while comparisons with alternative sensitivity frameworks highlight the interpretive advantages of the OVB decomposition.

Problem

• external validity
• omitted variable bias
• effect modification
• selection ignorability
• generalization

Innovation

• omitted variable bias
• sensitivity analysis
• external validity
• partial R-squared
• generalization of RCTs