randomized controlled trials

Designing and running experiments with random assignment of units to interventions so causal effects of manipulative or behavioral interventions can be estimated and tested while controlling for confounding and measuring outcomes.

randomizedcontrolledtrials

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

Synthetic Controls for Experimental Design

Aug 04, 2021
AA
Alberto Abadie
🏛️ MIT | Boston University

In large-scale aggregate-unit experiments (e.g., markets), conventional randomized treatment assignment often yields severe baseline imbalance due to extremely few treated units, leading to biased causal estimates. To address this, we systematically integrate the synthetic control method into experimental design, proposing a non-randomized treatment allocation mechanism: dynamically constructing a weighted synthetic control group based on pre-treatment covariates. We further develop配套 components—including counterfactual prediction, distance-driven unit matching, robust variance estimation, and a novel confidence interval construction procedure. Theoretically, our estimator is proven consistent and asymptotically normal. Empirically, it reduces estimation bias by 40–65% relative to standard randomization and substantially improves statistical power. Our core contribution is a new causal inference paradigm for small-N aggregate experiments—rigorous in inference, unbiased under mild assumptions, and highly interpretable.

Addresses experimental design for large aggregate unitsProposes synthetic control designs for accurate estimationReduces bias in treated and control group selection

This paper identifies a causal inference bias in service-intervention randomized controlled trials (RCTs) arising from provider capacity constraints: limited resources induce cross-participant interference and treatment dose heterogeneity, rendering the average treatment effect dependent on both sample size and capacity, and causing effect attenuation beyond a critical threshold—yielding non-monotonic, inverted-U-shaped statistical power. We formally characterize this capacity-constrained, queue-based interference mechanism for the first time, integrating queuing theory (square-root staffing rule), causal inference, and experimental design. Our method jointly optimizes provider capacity and sample size to maximize power under resource constraints. Results demonstrate substantial gains in statistical power, reduced required resources and participant enrollment, and provide a mechanistic explanation for the common phenomenon of intervention efficacy fading upon scaling—from RCT success to real-world failure.

Analyzing how capacity and sample size affect treatment effects and statistical powerExplaining replication failures in experiments due to operational dosage variationsModeling service interventions with capacity constraints using queueing theory

Experimentation for Homogenous Policy Change

Jan 28, 2021
MO
Molly Offer-Westort

This paper addresses causal inference under violations of the Stable Unit Treatment Value Assumption (SUTVA) and interference among units. Method: We propose the Homogeneous-Intervention Average Treatment Effect (HAATE) as a new target estimand for the Global Average Treatment Effect (GATE); formally define HAATE; prove theoretically that the difference-in-means estimator dominates a correctly specified regression model under interference; and design a two-stage cluster-randomized experiment that leverages intra-cluster treatment correlation to model cluster-level error, thereby substantially reducing root mean squared error (RMSE). Contribution/Results: Monte Carlo simulations and a large-scale online A/B test on Facebook demonstrate that, compared to conventional designs, our approach significantly improves estimation accuracy in finite samples—enhancing the reliability of policy-level causal inference under interference.

Comparing performance of estimators for Global Average Treatment EffectsEstimating treatment effects under interference among unitsEvaluating randomization designs for cluster-level correlated errors

Multiple Randomization Designs

Dec 27, 2021
LM
Lorenzo Masoero
🏛️ Amazon | University of Washington | Stanford University

Traditional randomized controlled trials (RCTs) suffer from cross-group spillovers and interference in multi-population interaction settings—e.g., buyer-seller or creator-subscriber systems—leading to biased estimation of the average treatment effect (ATE). This paper introduces the first systematic multidimensional randomization design framework, relaxing the conventional single-layer randomization assumption. By integrating hierarchical randomization, cross-group assignment, and potential outcomes modeling, it jointly identifies both the ATE and cross-group interference effects. We establish theoretical guarantees: the proposed unbiased estimator is consistent and asymptotically normal. Simulation results demonstrate substantially higher statistical power compared to standard RCTs. This work extends the scope of causal questions addressable through experimental design and provides a rigorous foundation for causal inference in complex intervention environments—particularly platform economies—where interdependent user behaviors induce non-negligible interference.

Addressing interference effects in multi-population experimental settingsDeveloping statistical methods for analyzing multiple randomization designsProposing new designs for experiments with interacting populations

Combining Experimental and Observational Data to Estimate Treatment Effects on Long Term Outcomes

Jun 17, 2020
SA
S. Athey
🏛️ Stanford University | Harvard University | NBER

This study addresses selection bias in estimating long-term causal effects—such as graduation rates—from observational studies. We propose a novel control function approach that leverages experimental estimates of treatment effects on short-term outcomes (e.g., eighth-grade test scores) to correct for unobserved confounding in large-scale administrative observational data. Our method integrates insights from difference-in-differences estimation, covariate balancing, and cross-sample effect calibration, enabling the first systematic correction based on heterogeneity in short-term treatment effects. By bridging randomized experiments and observational datasets, the framework jointly preserves internal validity from experiments and external representativeness from administrative records, overcoming inferential limitations inherent to single-data-source designs. Empirical validation using the STAR randomized experiment and New York State school administrative data demonstrates substantial improvements in both accuracy and external validity of estimated causal effects of class size on academic performance.

Correcting selection bias in observational studies via experimental dataDeveloping a method to weaken assumptions for surrogate estimatorsEstimating treatment effects on primary outcomes using observational and experimental data

Latest Papers

What's happening recently
View more

This study addresses the problem of testing whether a treatment effect operates entirely through observed mediators and identifying causal mechanisms under control for covariates. The authors propose a statistical test based on double machine learning, extending— for the first time—the joint evaluation of full mediation and causal mechanism identification to non-randomized treatment settings. By integrating conditional independence testing, the method achieves root-n consistent and asymptotically normal inference even in the presence of high-dimensional covariates. Simulation studies demonstrate favorable finite-sample performance, and the approach is successfully applied to two randomized experiments examining maternal mental health and social norms.

causal mechanismsfull mediationidentifiability

This study addresses the challenge of efficiently estimating causal effects under confounding when experimental budgets are limited. The authors propose a novel approach that integrates instrumental variable regression with Gaussian graphical models, leveraging prior knowledge of partial joint distributions to optimize the allocation between fully observed samples and partially observed data (e.g., only \(X_{12}\)). Under a fixed budget constraint, this method analytically derives the optimal sampling scheme that minimizes the asymptotic variance of the causal effect estimator—a solution not previously available in closed form. Theoretical analysis demonstrates that the proposed allocation significantly reduces both the total budget and the number of complete observations required to detect non-zero causal effects. Empirical validation in automotive analytics and drug discovery underscores the method’s practical utility alongside its theoretical contributions.

budget constraintcausal effect estimationexperimental design

Estimating Total Effects in Bipartite Experiments with Spillovers and Partial Eligibility

Nov 14, 2025
AT
Albert Tan
🏛️ Amazon | Stanford University

This paper addresses causal effect estimation in bilateral systems where some units lack treatment eligibility yet remain subject to interference. Recognizing that conventional methods—by ignoring interference—yield biased estimates and even sign reversals, we formally introduce the “partially admissible” bipartite experiment design. We propose two novel causal estimands: the Primary Total Treatment Effect (PTTE), capturing the aggregate impact on eligible units, and the Secondary Total Treatment Effect (STTE), quantifying the net effect on ineligible units. Methodologically, we develop an interference-aware ensemble estimator: leveraging exposure mappings and generalized propensity scores, we employ projection mapping under linear edge assumptions to precisely link treatment-side exposures to outcome-side responses, and incorporate a deterministic aggregation scheme to enhance estimation efficiency for sparse treatment-side data. Simulation and real-world field experiments demonstrate that our framework substantially reduces both bias and variance, effectively corrects pre-specified metric distortions induced by interference, and—in practical applications—reverses both statistical significance and sign of key decision-relevant metrics.

Addressing bias from interference when only subset of units receives treatmentDeveloping interference-aware estimators combining exposure mappings and machine learningEstimating causal effects in bipartite experiments with spillovers and partial eligibility

Adaptive Data-Borrowing for Improving Treatment Effect Estimation using External Controls

Aug 05, 2025
QY
Qinwei Yang
🏛️ Beijing Technology and Business University | National University of Singapore

Small-sample randomized controlled trials (RCTs) often suffer from low statistical efficiency, leading to imprecise treatment effect estimation. To address bias arising from insufficient comparability when borrowing external control data, this paper proposes an influence-function-based adaptive borrowing method: it quantifies the exchangeability between external controls and the RCT via a formal exchangeability hypothesis test, and determines data-driven optimal weights by balancing bias and variance. Theoretically, the approach integrates semiparametric efficient estimation with asymptotic analysis to ensure consistency and efficiency of the estimator. Simulation studies and real-data analyses demonstrate that the method substantially improves estimation precision while maintaining unbiasedness, outperforming existing strategies for integrating external controls.

Addressing bias from non-comparable external control samplesImproving RCT treatment effect estimation with external controlsOptimizing data-borrowing via bias-variance trade-off analysis

This study addresses the challenge of identifying heterogeneous treatment effects while controlling the false discovery rate (FDR) in matched observational studies, where limited sample sizes and unmeasured confounding pose significant obstacles. The authors propose a novel method that adaptively discovers interpretable subgroups defined by covariate thresholds under many-to-one matching designs. Their approach achieves exact FDR control at the subgroup level for the first time and integrates sensitivity analysis models to account for unobserved confounding, leveraging multiple controls to enhance statistical power. Theoretical analysis, simulations, and empirical evaluation demonstrate that the method outperforms existing baselines in both accuracy and power when estimating heterogeneous economic returns to college education.

effect modificationfalse discovery ratematched controls

Hot Scholars

LH

Leonhard Held

Professor of Biostatistics, University of Zurich
StatisticsBiostatisticsEpidemiology
LP

Lukas Pin

PhD Student Biostatistics, University of Cambridge
Adaptive DesignsNonparametric StatisticsResponse-adaptive Randomisation
MV

Mark van der Laan

Jiann-Ping Hsu/Karl E. Peace Professor of Biostatistics & Statistics, University of California Berkeley
StatisticsBiostatisticsCausal InferenceMachine Learning
SY

Shu Yang

North Carolina State University
Causal inference and missing data analysis
TJ

Thomas Jaki

Professor of Statistics, University of Regensburg and University of Cambridge
Pre-clinical StatisticsEarly Phase Clinical TrialsAdaptive Designs