Covariate-moderated Empirical Bayes Matrix Factorization

2025-05-16

AI Summary

Existing matrix factorization (MF) methods struggle to effectively integrate heterogeneous auxiliary information—such as images, text, and graphs—and rely on strong parametric assumptions about prior distributions. To address this, we propose cEBMF, the first modular empirical Bayesian matrix factorization framework. cEBMF supports arbitrary modelable side information, automatically learns flexible, data-driven probabilistic priors, and seamlessly unifies variational autoencoding with empirical Bayesian inference—while remaining compatible with both neural networks and classical statistical models. Crucially, it eliminates the need for prespecified prior forms, significantly improving low-rank structure estimation accuracy. Experiments on spatial transcriptomics and MovieLens datasets demonstrate that cEBMF achieves superior robustness and generalization compared to state-of-the-art methods. By enabling principled, scalable integration of multi-source heterogeneous covariates, cEBMF establishes a general, extensible paradigm for Bayesian matrix factorization under complex auxiliary information.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Reasoning under Uncertainty: Relational Probabilistic Models
• Data Mining & Knowledge Management: Recommender Systems

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Matrix factorization is a fundamental method in statistics and machine learning for inferring and summarizing structure in multivariate data. Modern data sets often come with ``side information'' of various forms (images, text, graphs) that can be leveraged to improve estimation of the underlying structure. However, existing methods that leverage side information are limited in the types of data they can incorporate, and they assume specific parametric models. Here, we introduce a novel method for this problem, covariate-moderated empirical Bayes matrix factorization (cEBMF). cEBMF is a modular framework that accepts any type of side information that is processable by a probabilistic model or neural network. The cEBMF framework can accommodate different assumptions and constraints on the factors through the use of different priors, and it adapts these priors to the data. We demonstrate the benefits of cEBMF in simulations and in analyses of spatial transcriptomics and MovieLens data.

Problem

• Incorporates diverse side information for matrix factorization
• Adapts priors to data without parametric assumptions
• Enhances structure estimation in multivariate data analysis

Innovation

• Modular framework for side information integration
• Adapts priors to data using empirical Bayes
• Accommodates various constraints through flexible priors