Learning Clinical-Trial Strategy: Offline Policy Training for Decision Agents

2026-08-04

AI Summary

This study addresses the sequential decision-making problem in clinical trial portfolio planning for oncology drug development by formulating it as an offline reinforcement learning task. Leveraging historical multi-source heterogeneous data, the authors construct, for the first time, a large-scale time-series clinical development dataset comprising 881 decision segments and propose a large language model (LLM) agent enhanced with a date-gated retrieval mechanism. The agent is trained using various offline policy learning strategies—including behavioral cloning, reward-weighted behavioral cloning, reward learning, and implicit Q-learning—with reward-weighted behavioral cloning yielding the best performance. It achieves 46.2% on the indication F1 metric and 14.2% on the strict F1 metric, significantly outperforming the unfinetuned baseline. These results demonstrate the effectiveness of structured offline learning for clinical trial planning.

Abstract

Clinical development is sequential decision-making under uncertainty, where a sponsor must plan a portfolio of experiments from heterogeneous evidence. We study this setting by framing oncology clinical development as an offline decision-making problem in which an agent predicts the next six-month trial portfolio of an oncology drug program from information available at the decision date. To support this, we construct a temporal dataset that combines 31.7k heterogeneous public data records, including trial registries, regulatory reviews, sponsor filings, utilization data, and epidemiology, into 881 offline decision episodes across 45 historical programs. We compare four offline objectives: behavioral cloning, reward-weighted behavioral cloning, learned-reward training, and value-based implicit Q-learning against four frontier LLM agents that share a common date-gated retrieval scaffold across held-out drug, sponsor, drug-class, and temporal splits. Models trained offline outperform the non-fine-tuned baselines, particularly in the post-August 2025 contamination-clean holdout. Reward-weighted behavioral cloning performs the best, obtaining 46.2% indication F1 and 14.2% strict F1 against 25.0% and 2.1%, respectively, for the best-performing tool agent on each metric. These results suggest that structured offline learning can teach agents to plan clinical experiments.

Problem

• clinical trial planning
• offline decision-making
• sequential decision-making under uncertainty
• oncology drug development
• policy learning

Innovation

• offline reinforcement learning
• clinical trial planning
• reward-weighted behavioral cloning
• heterogeneous data integration
• decision-making under uncertainty