🤖 AI Summary
This work addresses the limitations of the traditional Expectation–Maximization (EM) algorithm, which relies on ad hoc latent variable constructions and is restricted to specific missing-data settings, lacking a unified framework. The authors propose a Normalized EM (N-EM) algorithm that generalizes EM to log-likelihood optimization problems involving integral terms by introducing a normalized density function. This approach establishes a three-stage iterative scheme comprising a Normalization step (N-step), an Expectation step (E-step), and a Maximization step (M-step). For the first time, it provides a unified optimization framework applicable to a broader class of likelihood functions, eliminating the need for manually specified latent variables. The method not only solves problems intractable to conventional EM but also achieves efficient and consistent optimization in comparable scenarios. Theoretical analysis and extensive experiments confirm the convergence and effectiveness of the proposed algorithm.
📝 Abstract
Although the $\textit{expectation-maximization}$ (EM) algorithm is a powerful optimization tool in statistics, it can only be applied to missing/incomplete data problems or to problems with a latent-variable structure. It is well known that the introduction of latent variables (or the data augmentation) is an art; i.e., it could only be done case by case. In this paper, we propose a new algorithm, a so-called $\textit{normalized EM}$ (N-EM) algorithm, for a class of log-likelihood functions with integrals. As an extension of the original EM algorithm, the N-EM algorithm inherits all advantages of EM-type algorithms and consists of three steps: normalization step (N-step), expectation step (E-step) and maximization step (M-step), where the N-step is to construct a $\textit{normalized density function}$ (ndf), the E-step is to compute a well-established surrogate $Q$-function and the M-step is to maximize the $Q$-function as in the original EM algorithm. The ascent property, the best choice of the ndf, and those N-EM algorithms with a difficult M-step are also explored. By multiple real applications, we have shown that the N-EM algorithm can solve some problems which cannot be addressed by the EM algorithm. Next, for problems to which the EM can be applied (often case by case), the N-EM algorithm can be employed in a unified framework. Numerical experiments are performed and convergence properties are also established.