Sequential Control of False Positives in Online Change Point Detection

2026-07-16

AI Summary

This study addresses the challenge of multiple testing in online change-point detection, where repeated hypothesis tests render traditional family-wise error rate (FWER) control inadequate. The work introduces the first formal definition of sequential FWER (sFWER) tailored to real-time data streams and proposes a simulation-based dynamic threshold calibration mechanism that effectively controls the probability of at least one false alarm within a sliding monitoring window. By integrating sequential inference with a moving window framework, the method operates without requiring a pre-specified number of tests and is well-suited for highly dependent data. Empirical evaluations demonstrate that the proposed approach accurately controls sFWER and substantially outperforms existing methods, with successful application to passive smartphone sensing data from adolescents exhibiting emotional instability.

Abstract

Online change point detection is the process of identifying distributional changes in time-ordered data in real time. In applications such as mobile health (mHealth), repeated testing is often performed as new data arrive, creating a multiple testing problem. Traditional approaches for controlling the family-wise error rate (FWER) are not well suited to this setting because the tests are highly dependent and the number of tests is not fixed in advance. In this work, we introduce a sequential family-wise error rate (sFWER), defined as the probability of at least one false positive within a moving monitoring window. We propose a simulation-based calibration procedure to estimate monitoring thresholds that control the sFWER at a desired level. Through simulation studies, we demonstrate that the proposed procedure achieves the desired error control, while commonly used alternatives are either overly conservative or fail to adequately control false alarms. Finally, we illustrate the proposed approach using passively collected smartphone data from a cohort of adolescents and young adults with affective instability.

Problem

• online change point detection
• false positives
• multiple testing
• family-wise error rate
• sequential testing

Innovation

• sequential FWER
• online change point detection
• multiple testing
• simulation-based calibration
• false positive control