Leveraging External Data for Testing Experimental Therapies with Biomarker Interactions in Randomized Clinical Trials

2025-06-04

AI Summary

Clinical oncology trials frequently yield false-negative results due to treatment effect heterogeneity across patient subgroups—especially when subgroup analyses are not prespecified in the trial design. To address this, we propose a novel post-hoc subgroup efficacy testing framework that leverages external data (e.g., published clinical studies and electronic health records) to enhance statistical power. Our method employs a permutation-based inference procedure that requires no strong modeling assumptions, rigorously controls Type I error under arbitrary unmeasured confounding and population distribution shift, and achieves theoretical optimality in power. Evaluated on a multicenter retrospective glioblastoma study and extensive simulations, the approach significantly improves detection sensitivity for heterogeneous treatment effects, thereby mitigating decision distortion caused by effect heterogeneity. It provides a generalizable, statistically principled tool for re-evaluating trial null findings.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In oncology the efficacy of novel therapeutics often differs across patient subgroups, and these variations are difficult to predict during the initial phases of the drug development process. The relation between the power of randomized clinical trials and heterogeneous treatment effects has been discussed by several authors. In particular, false negative results are likely to occur when the treatment effects concentrate in a subpopulation but the study design did not account for potential heterogeneous treatment effects. The use of external data from completed clinical studies and electronic health records has the potential to improve decision-making throughout the development of new therapeutics, from early-stage trials to registration. Here we discuss the use of external data to evaluate experimental treatments with potential heterogeneous treatment effects. We introduce a permutation procedure to test, at the completion of a randomized clinical trial, the null hypothesis that the experimental therapy does not improve the primary outcomes in any subpopulation. The permutation test leverages the available external data to increase power. Also, the procedure controls the false positive rate at the desired $alpha$-level without restrictive assumptions on the external data, for example, in scenarios with unmeasured confounders, different pre-treatment patient profiles in the trial population compared to the external data, and other discrepancies between the trial and the external data. We illustrate that the permutation test is optimal according to an interpretable criteria and discuss examples based on asymptotic results and simulations, followed by a retrospective analysis of individual patient-level data from a collection of glioblastoma clinical trials.

Problem

• Testing experimental therapies with biomarker interactions in randomized trials
• Addressing false negatives due to heterogeneous treatment effects in subgroups
• Using external data to improve power without restrictive assumptions

Innovation

• Uses external data to enhance clinical trial power
• Introduces permutation test for heterogeneous treatment effects
• Controls false positive rate without restrictive assumptions