Multilevel Primary Aim Analyses of Clustered SMARTs: With Applications in Health Policy

2025-03-12

AI Summary

Marginal mean comparison for nested longitudinal outcomes (e.g., student–teacher–school three-level structures) in cluster-randomized sequential multiple assignment randomized trials (SMARTs) remains methodologically challenging under conventional two-level analytical frameworks. Method: We propose a novel three-level marginal mean modeling and estimation framework tailored to nested longitudinal SMART data, integrating three-level generalized estimating equations (GEE) with marginal mean modeling techniques. Contribution/Results: This approach substantially improves statistical efficiency and enhances detection of cluster-level adaptive intervention effects while preserving policy interpretability. Empirically validated in the ASIC school mental health policy optimization study, it demonstrates robust performance in real-world settings. The framework establishes a new paradigm for evaluating complex health policies under multilevel nested designs and provides a generalizable methodological toolkit for causal inference in clustered, adaptive interventions.

Technology Category

• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality

Application Category

• User Modeling, Personalization and Recommendation: Studies of user behavior, including longitudinal effects of personalized systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

In many health policy settings, adaptive interventions target a population of clusters (e.g., schools), with the ultimate intent of impacting outcomes at the level of individuals within the clusters. Health policy researchers can use clustered, sequential, multiple assignment, randomized trials (SMARTs) to answer important scientific questions concerning clustered adaptive interventions. A common primary aim is to compare the mean of a nested, end-of-study outcome between two clustered adaptive interventions. However, existing methods are not suitable when the primary outcome in a clustered SMART is nested and longitudinal (e.g., repeated outcome measures nested within mental healthcare providers, and mental healthcare providers nested within schools). This manuscript proposes a three-level marginal mean modeling and estimation approach for comparing adaptive interventions in a clustered SMART. The proposed method enables policy analysts to answer a wider array of scientific questions in the marginal comparison of clustered adaptive interventions. Further, relative to using an existing two-level method with a nested end-of-study outcome, the proposed method benefits from improved statistical efficiency. With this approach, we examine longitudinal comparisons of adaptive interventions for improving school-based mental healthcare and contrast its performance with existing approaches for studying static end-of-study outcomes. Methods were motivated by the Adaptive School-Based Implementation of CBT (ASIC) study, a clustered SMART designed to construct an adaptive health policy to improve the adoption of evidence-based CBT by mental healthcare professionals in high schools across Michigan.

Problem

• Develops three-level model for clustered SMARTs
• Enables longitudinal comparisons of adaptive interventions
• Improves efficiency over existing two-level methods

Innovation

• Three-level marginal mean modeling approach
• Longitudinal nested outcome analysis method
• Improved statistical efficiency over existing