Evaluation of clinical utility in emulated clinical trials

2025-06-04

AI Summary

Comparing the clinical utility of dynamic treatment regimes (DTRs) against current physician practice or guideline-recommended standard care is challenging when randomized clinical trials are infeasible. Method: We propose the first simulation-based clinical trial framework for observational data, integrating inverse probability weighting, doubly robust estimation, and sequential causal modeling to accommodate multi-stage treatment decisions. We formally define and construct multiple clinically interpretable utility estimands grounded in DTR literature, explicitly incorporating clinical guidelines into the control arm. Contribution/Results: Evaluated on real-world rheumatoid arthritis data, our estimands demonstrate stability and practical utility, enabling quantification of average population-level outcome differences across competing treatment rules. This work bridges a critical methodological gap in causal evaluation of DTRs from observational studies and provides a generalizable inference paradigm for guideline refinement and personalized treatment optimization.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Sequential Decision Making
• Machine Learning: Causal Learning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Dynamic treatment regimes have been proposed to personalize treatment decisions by utilizing historical patient data, but optimization can only be done over information available in the database. In contrast, the standard of care or physicians' decisions may be complex algorithms based on information that is not available to researchers. It is thus meaningful to integrate the standard of care into the evaluation of treatment strategies, and previous works have suggested doing so through the concept of clinical utility. Here we will focus on the comparative component of clinical utility as the average outcome had the full population received treatment based on the proposed dynamic treatment regime in comparison to the full population receiving the"standard"treatment assignment mechanism, such as a physician's choice. Clinical trials to evaluate clinical utility are rarely conducted, and thus, previous works have proposed an emulated clinical trial framework using observational data. However, only one simple estimator was previously suggested, and the practical details of how one would conduct this emulated trial were not detailed. Here, we illuminate these details and propose several estimators of clinical utility based on estimators proposed in the dynamic treatment regime literature. We illustrate the considerations and the estimators in a real data example investigating treatment rules for rheumatoid arthritis, where we highlight that in addition to the standard of care, the current medical guidelines should also be compared to any"optimal"decision rule.

Problem

• Evaluate clinical utility in emulated trials for personalized treatment
• Compare dynamic treatment regimes with standard care using observational data
• Propose new estimators for clinical utility in rheumatoid arthritis treatment

Innovation

• Utilizes historical patient data for treatment personalization
• Proposes emulated clinical trial framework using observational data
• Compares standard care with optimal dynamic treatment regimes