Non-Negative Matrix Factorization Using Non-Von Neumann Computers

2025-11-29

AI Summary

Non-negative matrix factorization (NMF) is an NP-hard unsupervised learning problem with broad applications yet limited scalability on classical von Neumann architectures. Method: We propose Quadratic Unconstrained Binary Optimization (QUBO) and quartic real-integer optimization formulations tailored for Ising machines and entropy-computing hardware, pioneering the integration of entropy-based computing paradigms into NMF. Our end-to-end implementation runs on the Dirac-3 entropy-computing accelerator. We further design a hybrid solver framework that synergistically combines Dirac-3 with classical algorithms (e.g., scikit-learn). Contribution/Results: Experiments demonstrate that our hybrid approach significantly outperforms pure software-based NMF (scikit-learn) in both reconstruction accuracy and computational efficiency. On integer-valued matrices, Dirac-3 alone consistently surpasses Google’s CP-SAT solver in solution quality and speed. This work establishes a novel hardware–algorithm co-design paradigm for tackling NP-hard unsupervised learning tasks.

Technology Category

• Machine Learning: Quantum Machine Learning
• Search and Optimization: Mixed Discrete/Continuous Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Non-negative matrix factorization (NMF) is a matrix decomposition problem with applications in unsupervised learning. The general form of this problem (along with many of its variants) is NP-hard in nature. In our work, we explore how this problem could be solved with an energy-based optimization method suitable for certain machines with non-von Neumann architectures. We used the Dirac-3, a device based on the entropy computing paradigm and made by Quantum Computing Inc., to evaluate our approach. Our formulations consist of (i) a quadratic unconstrained binary optimization model (QUBO, suitable for Ising machines) and a quartic formulation that allows for real-valued and integer variables (suitable for machines like the Dirac-3). Although current devices cannot solve large NMF problems, the results of our preliminary experiments are promising enough to warrant further research. For non-negative real matrices, we observed that a fusion approach of first using Dirac-3 and then feeding its results as the initial factor matrices to Scikit-learn's NMF procedure outperforms Scikit-learn's NMF procedure on its own, with default parameters in terms of the error in the reconstructed matrices. For our experiments on non-negative integer matrices, we compared the Dirac-3 device to Google's CP-SAT solver (inside the Or-Tools package) and found that for serial processing, Dirac-3 outperforms CP-SAT in a majority of the cases. We believe that future work in this area might be able to identify domains and variants of the problem where entropy computing (and other non-von Neumann architectures) could offer a clear advantage.

Problem

• Solves NP-hard non-negative matrix factorization using energy-based optimization
• Explores non-von Neumann architectures like Dirac-3 for NMF computation
• Compares Dirac-3 performance against classical methods for matrix reconstruction

Innovation

• Uses energy-based optimization on non-von Neumann computers
• Formulates NMF as QUBO and quartic models for specialized hardware
• Combines Dirac-3 entropy computing with classical methods for better accuracy