General measures of effect size to calculate power and sample size for Wald tests with generalized linear models

2025-06-27

AI Summary

Power and sample-size calculations for Wald tests in generalized linear models (GLMs) often rely on model-specific assumptions (e.g., logistic regression) or require difficult-to-specify nuisance parameters, limiting practical utility in study design. Method: We propose two general, easily obtainable effect size metrics applicable to any GLM, requiring only basic design information—number of predictors, covariates, significance level, and approximate response distribution. We derive the first asymptotic relative error bound for Wald-based power approximations and validate finite-sample accuracy and robustness via comprehensive simulations. Contribution/Results: The method enables precise estimation of statistical power, required sample size, and minimal detectable effect size across diverse GLMs—including logistic, Poisson, and Gaussian models—with high accuracy and broad applicability. It substantially extends the scope and practicality of power analysis for GLMs in real-world study planning.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Power and sample size calculations for Wald tests in generalized linear models (GLMs) are often limited to specific cases like logistic regression. More general methods typically require detailed study parameters that are difficult to obtain during planning. We introduce two new effect size measures for estimating power, sample size, or the minimally detectable effect size in studies using Wald tests across any GLM. These measures accommodate any number of predictors or adjusters and require only basic study information. We provide practical guidance for interpreting and applying these measures to approximate a key parameter in power calculations. We also derive asymptotic bounds on the relative error of these approximations, showing that accuracy depends on features of the GLM such as the nonlinearity of the link function. To complement this analysis, we conduct simulation studies across common model specifications, identifying best use cases and opportunities for improvement. Finally, we test the methods in finite samples to confirm their practical utility.

Problem

• General effect size measures for Wald tests in GLMs
• Overcoming limitations of specific cases like logistic regression
• Providing practical guidance for power and sample size calculations

Innovation

• New effect size measures for Wald tests
• Applicable to any generalized linear model
• Requires only basic study information