Testing Effect Homogeneity and Confounding in High-Dimensional Experimental and Observational Studies

2026-02-23

AI Summary

This work proposes a unified framework to test the homogeneity of conditional average treatment effects (CATE) across multiple experimental and observational studies and to assess the sensitivity of effect estimates to unobserved confounding. Built upon double machine learning, the approach accommodates high-dimensional covariates and is applicable to locally identified settings such as randomized controlled trials, instrumental variables, and difference-in-differences designs. It is the first framework to jointly integrate CATE homogeneity testing with robustness evaluation against unmeasured confounding, thereby enabling data-driven judgments about the validity of extrapolation. Simulations demonstrate favorable finite-sample performance, and an application to the International Stroke Trial (IST) provides empirical evidence supporting both the generalizability of causal effects and the plausibility of identification assumptions.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Intelligent Robots: State Estimation

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Large-scale security measurements

Abstract

We propose a framework for testing the homogeneity of conditional average treatment effects (CATEs) across multiple experimental and observational studies. Our approach leverages multiple randomized trials to assess whether treatment effects vary with unobserved heterogeneity that differs across trials: if CATEs are homogeneous, this indicates the absence of interactions between treatment and unobservables in the mean effect. Comparing CATEs between experimental and observational data further allows evaluation of potential confounding: if the estimands coincide, there is no unobserved confounding; if they differ, deviations may arise from unobserved confounding, effect heterogeneity, or both. We extend the framework to settings with alternative identification strategies, namely instrumental variable settings and panel data with parallel trends assumptions based on differences in differences, where effects are identified only locally for subpopulations such as compliers or treated units. In these contexts, testing homogeneity is useful for assessing whether local effects can be extrapolated to the total population. We suggest a test based on double machine learning that accommodates high-dimensional covariates in a data-driven way and investigate its finite-sample performance through a simulation study. Finally, we apply the test to the International Stroke Trial (IST), a large multi-country randomized controlled trial in patients with acute ischaemic stroke that evaluated whether early treatment with aspirin altered subsequent clinical outcomes. Our methodology provides a flexible tool for both validating identification assumptions and understanding the generalizability of estimated treatment effects.

Problem

• treatment effect homogeneity
• unobserved confounding
• conditional average treatment effects
• high-dimensional data
• causal inference

Innovation

• conditional average treatment effects
• effect homogeneity
• unobserved confounding
• double machine learning
• high-dimensional covariates