ALPACA: A Reinforcement Learning Environment for Medication Repurposing and Treatment Optimization in Alzheimer's Disease

2026-02-22

AI Summary

This work addresses the challenge of efficiently evaluating personalized sequential therapies for Alzheimer’s disease (AD), which is hindered by the disease’s prolonged progression and high patient heterogeneity. To this end, the authors propose ALPACA—an open-source, Gym-compatible reinforcement learning environment that integrates a Conditional Autoregressive State Transition (CAST) model with reinforcement learning, leveraging longitudinal data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). ALPACA constitutes the first interpretable and reusable digital trial platform for AD, enabling drug repurposing exploration and individualized treatment optimization while generating clinically plausible conditional treatment trajectories. Reinforcement learning policies trained within ALPACA significantly outperform both no-treatment and behavior cloning baselines on memory-related outcomes, with their decision-making grounded in clinically meaningful patient features.

Technology Category

• Machine Learning: Reinforcement Learning
• Multiagent Systems: Adversarial Agents
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Evaluating personalized, sequential treatment strategies for Alzheimer's disease (AD) using clinical trials is often impractical due to long disease horizons and substantial inter-patient heterogeneity. To address these constraints, we present the Alzheimer's Learning Platform for Adaptive Care Agents (ALPACA), an open-source, Gym-compatible reinforcement learning (RL) environment for systematically exploring personalized treatment strategies using existing therapies. ALPACA is powered by the Continuous Action-conditioned State Transitions (CAST) model trained on longitudinal trajectories from the Alzheimer's Disease Neuroimaging Initiative (ADNI), enabling medication-conditioned simulation of disease progression under alternative treatment decisions. We show that CAST autoregressively generates realistic medication-conditioned trajectories and that RL policies trained in ALPACA outperform no-treatment and behavior-cloned clinician baselines on memory-related outcomes. Interpretability analyses further indicated that the learned policies relied on clinically meaningful patient features when selecting actions. Overall, ALPACA provides a reusable in silico testbed for studying individualized sequential treatment decision-making for AD.

Problem

• Alzheimer's disease
• treatment optimization
• medication repurposing
• personalized therapy
• sequential decision-making

Innovation

• reinforcement learning
• medication repurposing
• personalized treatment
• disease progression modeling
• Alzheimer's disease