Edgington's Method for Random-Effects Meta-Analysis Part II: Prediction

2025-10-15

AI Summary

In random-effects meta-analysis, inference on the mean effect alone is insufficient when substantial between-study heterogeneity exists; moreover, existing prediction-distribution-based methods often neglect uncertainty in heterogeneity estimation, leading to undercoverage of prediction intervals. To address this, we propose a novel method that integrates Edgington’s p-value combination approach with a generalized heterogeneity statistic, constructing prediction intervals via confidence distributions to explicitly account for heterogeneity uncertainty. Simulation studies demonstrate strong performance: the 95% prediction intervals achieve nominal coverage when the number of included studies is ≥3, and accurately capture skewness in effect estimates even with ≤20 studies. This work represents the first systematic integration of p-value combination techniques with formal heterogeneity uncertainty correction, substantially enhancing both the statistical reliability and clinical interpretability of prediction distributions in meta-analysis.

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Statistical inference about the average effect in random-effects meta-analysis has been considered insufficient in the presence of substantial between-study heterogeneity. Predictive distributions are well-suited for quantifying heterogeneity since they are interpretable on the effect scale and provide clinically relevant information about future events. We construct predictive distributions accounting for uncertainty through confidence distributions from Edgington's $p$-value combination method and the generalized heterogeneity statistic. Simulation results suggest that 95% prediction intervals typically achieve nominal coverage when more than three studies are available and effectively reflect skewness in effect estimates in scenarios with 20 or less studies. Formulations that ignore uncertainty in heterogeneity estimation typically fail to achieve correct coverage, underscoring the need for this adjustment in random-effects meta-analysis.

Problem

• Predicting future effects in random-effects meta-analysis
• Accounting for uncertainty in heterogeneity estimation
• Developing accurate prediction intervals with limited studies

Innovation

• Predictive distributions using Edgington's p-value combination method
• Incorporating generalized heterogeneity statistic for uncertainty adjustment
• Achieving nominal coverage with prediction intervals for small studies