MM-algorithms for traditional and convex NMF with Tweedie and Negative Binomial cost functions and empirical evaluation

2026-03-10

AI Summary

This work addresses the limitations of traditional non-negative matrix factorization (NMF) methods, which typically assume Gaussian or Poisson noise and struggle to model over-dispersed data or complex mean–variance relationships. The authors propose a unified framework that extends both standard and convex NMF to negative binomial and Tweedie distributions, deriving corresponding multiplicative update rules via the Majorize-Minimisation (MM) algorithm. They introduce novel update rules for convex NMF under Poisson and negative binomial divergences—the first of their kind—and provide the first open-source implementation supporting multiple convex NMF variants. The study also establishes a theoretical connection between Tweedie distributions and β-divergences. Experimental results demonstrate that appropriate noise modeling significantly improves both data fitting and feature recovery, with convex NMF exhibiting notable efficiency and robustness when the number of classes is large.

Technology Category

Machine Learning: Matrix & Tensor MethodsSearch and Optimization: Non-convex OptimizationReasoning under Uncertainty: Relational Probabilistic Models

Application Category

Web Mining and Content Analysis: Robustness and generalizability of Web mining methodsGraph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsUser Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Non-negative matrix factorisation (NMF) is a widely used tool for unsupervised learning and feature extraction, with applications ranging from genomics to text analysis and signal processing. Standard formulations of NMF are typically derived under Gaussian or Poisson noise assumptions, which may be inadequate for data exhibiting overdispersion or other complex mean-variance relationships. In this paper, we develop a unified framework for both traditional and convex NMF under a broad class of distributional assumptions, including Negative Binomial and Tweedie models, where the connection between the Tweedie and the $β$-divergence is also highlighted. Using a Majorize-Minimisation approach, we derive multiplicative update rules for all considered models, and novel updates for convex NMF with Poisson and Negative Binomial cost functions. We provide a unified implementation of all considered models, including the first implementations of several convex NMF models. Empirical evaluations on mutational and word count data demonstrate that the choice of noise model critically affects model fit and feature recovery, and that convex NMF can provide an efficient and robust alternative to traditional NMF in scenarios where the number of classes is large. The code for our proposed updates is available in the R package nmfgenr and can be found at https://github.com/MartaPelizzola/nmfgenr.

Problem

• Non-negative matrix factorisation
• Overdispersion
• Tweedie distribution
• Negative Binomial
• Convex NMF

Innovation

• Non-negative Matrix Factorization
• Convex NMF
• Majorize-Minimisation
• Tweedie distribution
• Negative Binomial