🤖 AI Summary
Clinical studies often exhibit systematic discrepancies between the sample and the target population, inducing extrapolation bias. This paper investigates the robustness of inverse probability sampling weighting (IPSW) under misspecified target populations: even with correctly specified models, IPSW yields systematic bias if the selected target population fails to represent the actual inferential population. Through simulation experiments across diverse real-world covariate distributions and selection mechanisms, we quantify how deviation of the target population from representativeness affects the estimation of the population average treatment effect (PATE). Results demonstrate that bias increases monotonically with the degree of target-population mismatch—and in severe cases, IPSW performs worse than unweighted estimation. To our knowledge, this is the first systematic study revealing that target-population selection constitutes a foundational design decision in causal extrapolation, whose impact can surpass that of model misspecification—providing a critical methodological warning for causal inference beyond the study sample.
📝 Abstract
Clinical study populations often differ meaningfully from the broader populations to which results are intended to generalize. Weighting methods such as inverse probability of sampling weights (IPSW) reweight study participants to resemble a target population, but the accuracy of these estimates depends heavily on how well the chosen population represents the population of substantive interest. We conduct a simulation study grounded in empirical covariate distributions from several real-world data sources spanning a continuum from highly selective to broadly inclusive populations. Using treatment effect scenarios with varying levels of effect modification, we evaluate IPSW estimators of the population average treatment effect (PATE) across multiple candidate target populations. We quantify the bias that arises when the dataset used to operationalize the target population differs from the intended inference population, even when IPSW is correctly specified. Our results show that bias increases systematically as target populations diverge from a well-representative population, and that weighting to a poorly aligned target can introduce more bias than not weighting at all. These findings highlight that selecting an appropriate target population dataset is a critical design choice for valid generalization.