Factors affecting power in stepped wedge trials when the treatment effect varies with time

2025-03-14

AI Summary

In stepped-wedge cluster randomized trials (SW-CRTs), conventional instantaneous treatment (IT) models—assuming constant, immediate treatment effects—severely underestimate statistical power when treatment effects vary over time. This study systematically evaluates how design features, estimands (time-averaged treatment effect, TATE, versus period-specific treatment effect, PTE), and analytical models (extended treatment effect model, ETI, versus IT) jointly influence power under time-varying effects. Through large-scale simulation and validated power software, we quantify for the first time the sample size inflation factor required when switching from IT to ETI: achieving 90% power for full-period TATE necessitates 2.5–3× larger samples. Narrowing the TATE time window or increasing baseline/early-period observations substantially improves power, whereas extending the final period yields negligible gains. Results reveal SW-CRT’s inherent efficiency advantage for estimating short-term effects, providing methodological guidance for designing and analyzing SW-CRTs in dynamic-effect settings.

Technology Category

• Machine Learning: Ensemble Methods
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Causality

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Stepped wedge cluster randomized trials (SW-CRTs) have historically been analyzed using immediate treatment (IT) models, which assume the effect of the treatment is immediate after treatment initiation and subsequently remains constant over time. However, recent research has shown that this assumption can lead to severely misleading results if treatment effects vary with exposure time, i.e. time since the intervention started. Models that account for time-varying treatment effects, such as the exposure time indicator (ETI) model, allow researchers to target estimands such as the time-averaged treatment effect (TATE) over an interval of exposure time, or the point treatment effect (PTE) representing a treatment contrast at one time point. However, this increased flexibility results in reduced power. In this paper, we use public power calculation software and simulation to characterize factors affecting SW-CRT power. Key elements include choice of estimand, study design considerations, and analysis model selection. For common SW-CRT designs, the sample size (individuals per cluster-period) must be increased by a factor of roughly 2.5 to 3 to maintain 90% power when switching from an IT model to an ETI model (targeting the TATE over the entire study). However, the inflation factor is lower when considering TATE estimands over shorter periods that exclude longer exposure times for which there is limited information. In general, SW-CRT designs (including the ``staircase'' variant) have much greater power for estimating ``short-term effects'' relative to ``long-term effects''. For an ETI model targeting a TATE estimand, substantial power can be gained by adding time points to the start of the study or increasing baseline sample size, but surprisingly little power is gained from adding time points to the end of the study. More restrictive choices for modeling the exposure time... [truncated]

Problem

• Stepped wedge trials lose power when modeling time-varying treatment effects
• Traditional immediate treatment models produce misleading results over time
• Sample size requirements increase significantly for accurate time-varying effect estimation

Innovation

• Using exposure time indicator model for time-varying effects
• Employing power calculation software and simulation methods
• Increasing sample size to maintain power with ETI model