Q-Learning with Clustered-SMART (cSMART) Data: Examining Moderators in the Construction of Clustered Adaptive Interventions

2025-05-01

AI Summary

This paper addresses the challenge of evaluating candidate effect modifiers in cluster-adaptive interventions (cAI), particularly when nonregularity undermines valid statistical inference. We propose a Q-learning framework tailored to clustered data, innovatively embedding an M-out-of-N cluster bootstrap within the Q-learning estimation procedure to enable robust causal inference for tail-effect modifiers—first such application. The method integrates clustered Q-learning, causal effect modification analysis, and robust confidence interval construction. Applied to the ADEPT real-world dataset, it successfully generated clinic-level cAI policies. Simulation studies demonstrate that the proposed method achieves confidence interval coverage close to the nominal level, markedly improving inferential reliability under challenging conditions—including small numbers of clusters and high intra-cluster correlation. Our approach provides a generalizable, causally grounded methodology for designing cAI.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

A clustered adaptive intervention (cAI) is a pre-specified sequence of decision rules that guides practitioners on how best - and based on which measures - to tailor cluster-level intervention to improve outcomes at the level of individuals within the clusters. A clustered sequential multiple assignment randomized trial (cSMART) is a type of trial that is used to inform the empirical development of a cAI. The most common type of secondary aim in a cSMART focuses on assessing causal effect moderation by candidate tailoring variables. We introduce a clustered Q-learning framework with the M-out-of-N Cluster Bootstrap using data from a cSMART to evaluate whether a set of candidate tailoring variables may be useful in defining an optimal cAI. This approach could construct confidence intervals (CI) with near-nominal coverage to assess parameters indexing the causal effect moderation function. Specifically, it allows reliable inferences concerning the utility of candidate tailoring variables in constructing a cAI that maximizes a mean end-of-study outcome even when"non-regularity", a well-known challenge exists. Simulations demonstrate the numerical performance of the proposed method across varying non-regularity conditions and investigate the impact of varying number of clusters and intra-cluster correlation coefficient on CI coverage. Methods are applied on ADEPT dataset to inform the construction of a clinic-level cAI for improving evidence-based practice in treating mood disorders.

Problem

• Evaluating causal effect moderation in clustered adaptive interventions
• Developing Q-learning with cSMART data for optimal intervention tailoring
• Assessing confidence intervals for moderation under non-regularity challenges

Innovation

• Q-learning with clustered-SMART data
• M-out-of-N Cluster Bootstrap method
• Handling non-regularity in causal moderation