Asymptotic regimes for maximum likelihood estimation in the Ewens--Pitman model: When the strength parameter matters

2026-06-11

AI Summary

This study addresses the long-overlooked role of the strength parameter θ in asymptotic analyses of the Ewens–Pitman model. By integrating maximum likelihood estimation, asymptotic statistical theory, and frequency spectrum analysis, it systematically investigates the asymptotic behavior of the maximum likelihood estimators for both the discount parameter α and θ under large-sample regimes. The work reveals, for the first time, four distinct asymptotic regimes governed by θ under different limiting behaviors of the frequency spectrum, thereby transcending the classical assumption of infinite exchangeability that accommodates only two restricted cases. Furthermore, the authors propose a scaled model allowing θ to vary with sample size, substantially enhancing modeling flexibility for empirical frequency spectra. Both theoretical guarantees and empirical evaluations demonstrate that the proposed framework effectively captures complex spectral structures inaccessible to conventional models.

Abstract

We study the large sample asymptotic behaviour of the Maximum Likelihood Estimator of the discount and strength parameters $(α,θ)$ in the Ewens--Pitman model for random partitions, under mild assumptions on the data-generating mechanism. We show that four distinct regimes arise, depending on the limiting behaviour of the frequency spectrum. In particular, in contrast with previous work, we find that $θ$ may play a crucial role asymptotically. We further show that the existing literature implicitly focuses on only two of these regimes, and we relate this restriction to the constraints imposed by infinite exchangeability. Under the latter, indeed, the number of distinct blocks and the frequency spectrum are necessarily tied by a rigid structural relation. We prove that this lack of flexibility can be overcome through what we call the scaled Ewens--Pitman model, in which $θ$ is allowed to grow with the sample size $n$. Finally, we provide empirical evidence from real-world data showing that such extensions are needed to capture frequency spectra that fall outside the classical Ewens--Pitman framework.

Problem

• Ewens--Pitman model
• maximum likelihood estimation
• asymptotic regimes
• strength parameter
• frequency spectrum

Innovation

• Ewens–Pitman model
• maximum likelihood estimation
• asymptotic regimes
• strength parameter
• scaled model