Edgington's Method for Random-Effects Meta-Analysis Part I: Estimation

2025-10-14

AI Summary

This study addresses the undercoverage of confidence intervals in random-effects meta-analysis caused by uncertainty in heterogeneity estimation, which compromises conventional combined p-value methods (e.g., Edgington’s method). We propose a novel framework integrating p-value functions with confidence distributions (CDs). By constructing a generalized heterogeneity statistic and leveraging the CD to calibrate sampling variability in the between-study variance estimator τ², our approach robustly adjusts the combined p-value function. The method achieves nominal 95% coverage—particularly for meta-analyses with ≥3 heterogeneous studies—while yielding smaller point estimation bias and narrower confidence intervals than established alternatives such as the Hartung–Knapp–Sidik–Jonkman procedure. Overall, it enhances the accuracy and reliability of inference across multiple studies.

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Meta-analysis can be formulated as combining $p$-values across studies into a joint $p$-value function, from which point estimates and confidence intervals can be derived. We extend the meta-analytic estimation framework based on combined $p$-value functions to incorporate uncertainty in heterogeneity estimation by employing a confidence distribution approach. Specifically, the confidence distribution of Edgington's method is adjusted according to the confidence distribution of the heterogeneity parameter constructed from the generalized heterogeneity statistic. Simulation results suggest that 95% confidence intervals approach nominal coverage under most scenarios involving more than three studies and heterogeneity. Under no heterogeneity or for only three studies, the confidence interval typically overcovers, but is often narrower than the Hartung-Knapp-Sidik-Jonkman interval. The point estimator exhibits small bias under model misspecification and moderate to large heterogeneity. Edgington's method provides a practical alternative to classical approaches, with adjustment for heterogeneity estimation uncertainty often improving confidence interval coverage.

Problem

• Extending meta-analysis framework with combined p-value functions
• Incorporating heterogeneity estimation uncertainty via confidence distributions
• Improving confidence interval coverage in random-effects meta-analysis

Innovation

• Extends Edgington's method with confidence distribution approach
• Adjusts for heterogeneity uncertainty using generalized statistic
• Provides improved confidence interval coverage in meta-analysis