🤖 AI Summary
Null Hypothesis Significance Testing (NHST) suffers from fundamental limitations, including conflation of statistical and practical significance, sensitivity to sample size, and inability to distinguish “failure to reject” from “acceptance” of the null hypothesis. This paper introduces REACT—a novel hypothesis testing framework that integrates Bayesian logic with frequentist interpretability. REACT employs a dual-threshold decision rule based on confidence intervals for effect sizes and the minimal effect size of interest (MES), enabling, for the first time, joint inference over multiple parameters without multiplicity correction. Crucially, it formally distinguishes “absence of evidence” from “evidence of absence.” Empirical evaluation across multiple real-world datasets demonstrates that REACT substantially enhances scientific robustness and reproducibility of inferences, while maintaining computational and operational complexity comparable to NHST—facilitating straightforward adoption by researchers.
📝 Abstract
While Null Hypothesis Significance Testing (NHST) remains a widely used statistical tool, it suffers from several shortcomings, such as conflating statistical and practical significance, sensitivity to sample size, and the inability to distinguish between accepting the null hypothesis and failing to reject it. Recent efforts have focused on developing alternatives to NHST to address these issues. Despite these efforts, conventional NHST remains dominant in scientific research due to its simplicity and perceived ease of interpretation. Our work presents a novel alternative to NHST that is just as accessible and intuitive: REACT. It not only tackles the shortcomings of NHST but also offers additional advantages over existing alternatives. For instance, REACT is easily applicable to multiparametric hypotheses and does not require stringent significance-level corrections when conducting multiple tests. We illustrate the practical utility of REACT through real-world data examples, using criteria aligned with common research practices to distinguish between the absence of evidence and evidence of absence.
