Statistical Design of Pragmatic Trials Using Electronic Health Record Data when Outcome Assessments are Uncontrolled and Irregular

2026-05-08

AI Summary

This study addresses the bias in treatment effect estimation that arises in pragmatic trials using electronic health records when outcome assessments are uncontrolled, irregular, and potentially influenced by the intervention itself. Leveraging pre-trial cohort data, the authors developed a tailored simulation framework to systematically compare single-timepoint approaches with longitudinal models in handling intervention-dependent assessment timing. By incorporating linear mixed models with exponential correlation structures, time-varying intervention effects, and flexible post-baseline timepoint selection to estimate either specific or average treatment effects, the study demonstrates that naive methods ignoring assessment timing dependencies yield substantial bias. In contrast, longitudinal models accommodating flexible follow-up schedules produce unbiased estimates, with the linear mixed model featuring an exponential correlation structure exhibiting optimal performance—providing a critical analytical foundation for pragmatic trials such as MI-CARE.

Abstract

Pragmatic trials increasingly define outcomes using real-world data such as electronic health records, where assessments are collected during routine care rather than at fixed timepoints. Consequently, these uncontrolled assessments may be irregular, sparse, and affected by the intervention (intervention-dependent assessments), which can lead to biased treatment effect estimates. We developed a simulation study to inform the statistical approach for trials with uncontrolled assessments, which we applied to the MI-CARE pragmatic trial. Using a pre-trial cohort mimicking eligibility and outcome measurement, we estimated assessment frequency and timing and combined these estimates with assumptions about how the intervention effects might impact assessment. We simulated sparse and intervention-dependent assessments and compared single-measure approaches with longitudinal models using all scores. Under intervention-dependent assessments, we found that naive methods such as using the best score or using a randomly selected score without adjusting for measurement timing produced substantial bias. Models that adjusted flexibly for the follow-up timing estimated time-point specific or time-averaged treatment effects without bias. Simulation results informed the selection of the statistical approach for the MI-CARE trial. Among unbiased methods, the most powerful was a linear mixed model with exponential correlation structure, adjustment for time since baseline, and a time-varying intervention effect to estimate the intervention effect at the end of the intervention window. Future studies can use pre-trial data to conduct a simulation study tailored to the trial's data features to inform the analytic approach. Trials with uncontrolled assessments should consider the potential for intervention-dependent assessments and select an appropriate method to avoid bias.

Problem

• pragmatic trials
• electronic health records
• uncontrolled assessments
• intervention-dependent assessments
• treatment effect bias

Innovation

• pragmatic trials
• electronic health records
• intervention-dependent assessments
• longitudinal modeling
• simulation-informed analysis