🤖 AI Summary
Traditional topic models impose a “sum-to-one” constraint on parameters, leading to complex optimization landscapes and low computational efficiency. This work proposes integrating nonnegative matrix factorization (NMF) into topic model parameter estimation, eliminating the hard simplex constraint and instead leveraging NMF’s nonnegativity and low-rank structure to implicitly model topic distributions—thereby substantially simplifying the optimization problem. Methodologically, we design an efficient solving framework built upon state-of-the-art NMF algorithms and incorporate a post-processing step to recover interpretable probabilistic parameters; the method is implemented in the R package *fastTopics*. Experiments demonstrate consistent improvements in both accuracy and speed under maximum likelihood estimation and variational inference: superior fit within fixed time budgets, or significantly reduced runtime at equivalent accuracy. To our knowledge, this is the first systematic integration of NMF’s optimization advantages across the entire topic modeling pipeline, offering a new paradigm for high-dimensional text modeling that balances theoretical simplicity with computational scalability.
📝 Abstract
We report on the potential for using algorithms for non-negative matrix factorization (NMF) to improve parameter estimation in topic models. While several papers have studied connections between NMF and topic models, none have suggested leveraging these connections to develop new algorithms for fitting topic models. NMF avoids the"sum-to-one"constraints on the topic model parameters, resulting in an optimization problem with simpler structure and more efficient computations. Building on recent advances in optimization algorithms for NMF, we show that first solving the NMF problem then recovering the topic model fit can produce remarkably better fits, and in less time, than standard algorithms for topic models. While we focus primarily on maximum likelihood estimation, we show that this approach also has the potential to improve variational inference for topic models. Our methods are implemented in the R package fastTopics.