Cressie Read Power Divergence for Moment-Based Estimation: Hyperparameter and Finite Sample Behavior

2026-03-23

AI Summary

This study addresses the common practice of arbitrarily fixing the power parameter in Cressie–Read power divergence for moment-based estimation, which lacks systematic understanding of its finite-sample implications. For the first time, this work explicitly treats the power parameter as a crucial hyperparameter governing the curvature of the objective function and the resulting estimator performance. Through second-order asymptotic analysis, generalized empirical likelihood theory, Monte Carlo simulations, and an empirical case study, the paper uncovers its dual influence on both structural parameter estimates and Lagrange multipliers. The findings demonstrate that the power parameter substantially affects estimation bias and confidence interval coverage, with practical benefits of tuning validated in Owen’s canonical example, thereby offering both theoretical justification and actionable guidance for enhancing estimation robustness and accuracy.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Intelligent Robots: State Estimation

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

We study Cressie Read power divergence (CRPD) estimation for moment based models, focusing on finite sample behavior. While generalized empirical likelihood estimators, dual to CRPD, are known to outperform generalized method of moments estimators in small to moderate samples, the power parameter is typically chosen arbitrarily by the researcher, serving mainly as an index. We interpret it as a hyperparameter that determines the loss function and governs the learning procedure, shaping the curvature of the objective and influencing finite sample performance. Using second order asymptotics, we show that it affects both the structural estimator and the associated Lagrange multipliers, governing robustness, bias, and sensitivity to sampling variation. Monte Carlo simulations illustrate how estimator performance varies with the choice of the power parameter and underlying distributional features, with implications for second order bias and coverage distortion. An empirical illustration based on Owen (2001)s classical example highlights the practical relevance of tuning the power parameter.

Problem

• Cressie-Read power divergence
• moment-based estimation
• hyperparameter
• finite sample behavior
• power parameter

Innovation

• Cressie-Read power divergence
• hyperparameter tuning
• finite sample behavior
• moment-based estimation
• second-order asymptotics