meta-analysis methods

Statistical procedures to combine heterogeneous estimates from multiple studies into a harmonized dataset or pooled inference, including bias adjustment, uncertainty propagation, and assessments of statistical reliability and effect heterogeneity across tasks or conditions.

meta-analysismethods

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

This paper addresses the challenge of simultaneously ensuring internal validity (from randomized controlled trials, RCTs) and external validity (from observational studies) in heterogeneous treatment effect (HTE) estimation. We propose a tunable multi-task Gaussian process (MTGP) framework grounded in Bayesian nonparametric modeling and data fusion theory. Unlike conventional joint modeling or weighted integration approaches, our method flexibly balances RCT unbiasedness and observational data representativeness by sharing a common covariance structure across tasks while incorporating task-specific bias terms. It enables principled borrowing of biased-control information across data sources and provides calibrated uncertainty quantification. In simulation studies and a real-world educational intervention trial, the method achieves significantly improved point estimation accuracy and nominal coverage of prediction intervals across the full covariate domain. This work establishes a new paradigm for causal extrapolation and individualized policy evaluation—rigorous in statistical foundations and practical in application.

Addresses internal-external validity trade-off in causal inference studiesEstimates heterogeneous treatment effects by fusing RCT and observational dataUses multi-task Gaussian processes for robust effect estimation and uncertainty quantification

Edgington's Method for Random-Effects Meta-Analysis Part I: Estimation

Oct 14, 2025
DK
David Kronthaler
🏛️ University of Zurich

This study addresses the undercoverage of confidence intervals in random-effects meta-analysis caused by uncertainty in heterogeneity estimation, which compromises conventional combined p-value methods (e.g., Edgington’s method). We propose a novel framework integrating p-value functions with confidence distributions (CDs). By constructing a generalized heterogeneity statistic and leveraging the CD to calibrate sampling variability in the between-study variance estimator τ², our approach robustly adjusts the combined p-value function. The method achieves nominal 95% coverage—particularly for meta-analyses with ≥3 heterogeneous studies—while yielding smaller point estimation bias and narrower confidence intervals than established alternatives such as the Hartung–Knapp–Sidik–Jonkman procedure. Overall, it enhances the accuracy and reliability of inference across multiple studies.

Extending meta-analysis framework with combined p-value functionsImproving confidence interval coverage in random-effects meta-analysisIncorporating heterogeneity estimation uncertainty via confidence distributions

This paper addresses heterogeneous treatment effect (HTE) estimation in right-censored survival data. We propose a novel integrative method that jointly leverages randomized controlled trial (RCT) data and real-world data (RWD), which suffer from unmeasured confounding, measurement error, and selection bias. Our key innovation is a unified confounding function that simultaneously characterizes these multiple sources of bias, enabling joint identification of HTE across data sources. Within a reproducing kernel Hilbert space (RKHS) framework, we develop a penalized spline estimator for the treatment–covariate interaction effect, ensuring theoretically guaranteed convergence of the integrated causal estimator. Simulation studies and analysis of non-small-cell lung cancer data demonstrate that our approach significantly improves HTE estimation accuracy and statistical efficiency over RCT-only methods—reducing root mean squared error (RMSE) by up to 32%. The method provides a robust, generalizable tool for covariate-dependent survival effect assessment in precision medicine.

Characterizing bias from unmeasured confounders and censoring mechanismsEstimating heterogeneous treatment effects with right-censored survival dataSynthesizing randomized trials and real-world data with biases

A novel decomposition to explain heterogeneity in observational and randomized studies of causality

Aug 10, 2022
BG
Brian Gilbert
🏛️ New York University Grossman School of Medicine | Columbia University | University Paris Est Creteil

This study addresses the inconsistency in causal effect estimates between observational studies and randomized controlled trials (RCTs) by proposing the first unified framework for decomposing causal effect heterogeneity. The framework systematically identifies and quantifies three sources of heterogeneity: differences in covariate distributions, variation in mediating pathways, and shifts in outcome-generating mechanisms. Methodologically, it formally defines effect decomposition across data types (observational vs. experimental), integrating causal inference, sensitivity analysis, and decomposition modeling, while enabling robust parameter estimation under multiple hypotheses. Evaluated through simulation studies and an empirical analysis of the “Moving to Opportunity” experiment, the framework demonstrates improved interpretability, robustness, and policy generalizability in synthesizing evidence from heterogeneous data sources.

Addressing differences in covariate distributions and mechanismsExplaining heterogeneity in causal effects across studiesIdentifying sources of variability in treatment effects

Combining an experimental study with external data: study designs and identification strategies

Jun 05, 2024
LU
Lawson Ung
🏛️ Harvard T.H. Chan School of Public Health | Dartmouth Geisel School of Medicine | Beth Israel Deaconness Medical Center

This study addresses the challenge of integrating randomized or single-arm clinical trials with external experimental or observational data to enable cross-study treatment comparisons and improve estimation precision of treatment effects. Methodologically, building upon the potential outcomes framework, we first develop a unified identification strategy for hybrid-data designs, systematically characterizing identifiability conditions across diverse designs—including historical controls, synthetic controls, and anchoring estimators—and propose a generalizable taxonomy of such designs along with corresponding causal inference principles. Our contribution lies in filling a critical theoretical gap in regulatory science regarding the rigorous integration of external controls, thereby establishing a methodological foundation for leveraging real-world evidence to complement trial-based evidence in pharmaceutical and medical device evaluation. This advancement significantly enhances the transportability of evidence and its applicability to regulatory decision-making.

Combining experimental studies with external data sourcesDeveloping identification strategies for treatment effectsFormalizing study designs to support systematic evaluation

Latest Papers

What's happening recently
View more

This study addresses the challenge of ensuring rigor in causal inference under multi-source heterogeneous data fusion by proposing a structured design paradigm grounded in the target trial framework. The approach explicitly incorporates the target population and its sampling model into the causal analysis, systematically integrating external controls, generalizability, and transportability assessments through data element alignment, transparent assumption articulation, and emulation of the target trial. Its key innovation lies in anchoring the entire framework to a precise definition of the target population, thereby identifying and mitigating irreconcilable conflicts across data sources. This strategy enhances both the reliability and interpretability of causal conclusions derived from complex, real-world data ecosystems.

causal inferencedata integrationexternal comparator analyses

Traditional meta-analysis methods often assume either complete homogeneity or complete heterogeneity across studies, which can lead to biased estimates or loss of efficiency. This work proposes a heterogeneity-adaptive meta-analysis framework that, within a linear model setting, shrinks individual study distributions toward a shared “centroid” via Kullback–Leibler (KL) divergence regularization, enabling geometrically natural and dynamic information sharing. By avoiding extreme assumptions about between-study variability, the method yields a closed-form estimator whose mean squared error is theoretically guaranteed to be strictly lower than that of the conventional maximum likelihood estimator. Simulation studies demonstrate the approach’s flexibility and robustness, while an application to real-world data from the eICU Collaborative Research Database further confirms its practical effectiveness.

heterogeneityinformation sharingKullback-Leibler divergence

This study addresses a critical limitation in conventional evaluation of combination therapies—the frequent neglect of heterogeneity in individual patient responses to single agents, which induces bias in treatment effect estimation. The authors propose a model-free statistical inference framework that, without imposing assumptions on functional forms, systematically tackles the non-identifiability and nonlinearity arising from such heterogeneity. They establish partial identifiability conditions for cross-world target parameters and introduce an outcome-based optimal matching strategy that achieves √N-consistent estimation and asymptotically valid confidence intervals. Reanalysis of the ACTG 175 trial demonstrates that this approach substantially enhances the accuracy, reliability, and transparency of efficacy assessment for combination therapies.

combination therapymonotherapy responsepatient-level heterogeneity

This study addresses the challenge of accurately inferring the distribution of individual treatment effects—such as the proportion benefiting, the median effect, or the maximum impact—in randomized experiments, without suffering power loss due to suboptimal pre-specified test statistics. The authors propose an adaptive randomization test that combines multiple rank-based statistics, ensuring finite-sample validity without requiring prior knowledge of the optimal statistic. Innovatively integrating adaptive statistic combination with stratified weighting, the method effectively circumvents the power degradation typically induced by multiple comparison corrections and accommodates heterogeneous stratified experimental designs. In an empirical application to a teacher training program, the approach reveals that approximately half of the teachers experience significant benefits, demonstrating superior detection power and interpretability compared to conventional single rank-based tests.

distributional inferenceindividual treatment effectsrandomization tests

Existing methods for heterogeneous treatment effect estimation often struggle to simultaneously ensure sensitivity in detecting effect modifiers and validity in statistical inference. This work proposes a hybrid algorithm that integrates significance-driven splitting with honest estimation: it employs the t² statistic as the splitting criterion, incorporates honest sample splitting, selects the cost-complexity penalty via cross-validation, and uses the infinitesimal jackknife to estimate Monte Carlo variance. This approach is the first to align significance-based splitting with an honest estimation framework, maintaining theoretical consistency under strong interactions and providing nominal leaf-level confidence intervals for a single tree. Empirical results demonstrate approximately 90% coverage (nominal 90%) on Athey–Imbens synthetic data and Qini coefficients on par with S- and T-learners on real-world Criteo and Starbucks datasets.

causal inferenceconfidence interval coverageeffect modification

Hot Scholars

LH

Leonhard Held

Professor of Biostatistics, University of Zurich
StatisticsBiostatisticsEpidemiology
MT

Mike Thelwall

School of Information, Journalism and Communication, The University of Sheffield
scientometricsaltmetricssentiment analysissocial media
JJ

Julie Josse

Senior Researcher Inria,
Missing valuesLow rank matrixcausal inferenceR
ZF

Zhichao Fang

School of Information Resource Management, Renmin University of China
AltmetricsScientometricsBig Data Analysis
CZ

Chengzhi Zhang

Nanjing University of Science and Technology
Text MiningNatural Language ProcessingScience of Science