Technology Category
Reasoning under Uncertainty: Other Foundations of Reasoning under UncertaintyGame Theory and Economic Paradigms: Coordination and CollaborationKnowledge Representation and Reasoning: Preferences
Directional replicability: when can the factor of two be omitted
🤖 AI Summary
Directional replicability studies address whether at least $ r geq 2 $ out of $ n $ independent studies exhibit effects in the same direction, without prespecifying the direction a priori. Existing methods combine two-sided p-values by merging left- and right-tailed p-values and doubling the result to correct for double-testing, yet this approach is overly conservative.
Method: Under the assumptions that effect sizes are symmetrically distributed about zero and studies are independent, we prove that the factor-of-two correction is unnecessary while still strictly controlling the family-wise error rate (FWER). We propose a correction-free directional consistency testing framework grounded in one-sided p-value combination and directional inference theory.
Results: The proposed method substantially improves statistical power—particularly in multi-center validation, meta-analysis, and replicability assessment—offering a more sensitive and theoretically rigorous tool for evaluating directional replicability.
Technology Category
Application Category
📝 Abstract
Directional replicability addresses the question of whether an effect studied across $n$ independent studies is present with the same direction in at least $r$ of them, for $r geq 2$. When the expected direction of the effect is not specified in advance, the state of the art recommends assessing replicability separately by combining one-sided $p$-values for both directions (left and right), and then doubling the smaller of the two resulting combined $p$-values to account for multiple testing. In this work, we show that this multiplicative correction is not always necessary, and give conditions under which it can be safely omitted.
Problem
Research questions and friction points this paper is trying to address.
Addresses directional replicability across multiple independent studies
Determines when effect direction consistency requires statistical correction
Identifies conditions allowing omission of multiplicative p-value adjustment
Innovation
Methods, ideas, or system contributions that make the work stand out.
Directional replicability analysis without multiplicative correction
Conditions for omitting factor of two correction
Combining one-sided p-values from both directions
🔎 Similar Papers
No similar papers found.
