On solving symmetric multi-type orthogonal non-negative matrix tri-factorization problem

2026-06-06

AI Summary

This study addresses the joint tri-factorization of symmetric multi-type nonnegative matrices, aiming to learn shared orthogonal nonnegative factors that facilitate interpretable clustering and network analysis. It introduces, for the first time, a shared orthogonal nonnegative factor structure that enhances interpretability while preserving hard assignment properties. To optimize this formulation, two efficient algorithms are proposed: one based on a penalty-function fixed-point method derived from KKT conditions, and another employing a three-stage strategy that integrates nonnegative optimization, orthogonalization, and constrained ADAM fine-tuning. Experimental results demonstrate that the proposed methods reliably recover near-optimal factorizations on noisy synthetic data and yield embeddings that match or outperform established baselines—including SVD and node2vec—in citation network benchmarks across link prediction, node clustering, and classification tasks.

Abstract

We study the symmetric multi-type orthogonal non-negative matrix tri-factorization problem, where several symmetric non-negative matrices are simultaneously approximated by factors of the form $GS_{i}G^{\top}$, with a shared non-negative and orthogonal factor $G$. This model is motivated by clustering and network analysis, where non-negativity improves interpretability and orthogonality gives a natural assignment-type structure to the latent factor. Since the resulting optimization problem is highly non-convex, we develop two heuristic algorithms for computing high-quality local solutions. The first one is a fixed point method derived from the Karush-Kuhn-Tucker conditions after adding a penalty term for the orthogonality constraint. The second one is a three-stage ADAM-based method that combines non-negativity-preserving optimization, orthogonalization, and restricted ADAM refinement on the feasible set. We evaluate both methods on synthetic data, including noisy instances, and on citation network benchmarks. The synthetic experiments show that both algorithms recover factorizations close to the optimum and remain stable under noise. On real networks, the learned embeddings are competitive with or better than standard baselines such as SVD, node2vec, and classical link prediction heuristics in link prediction, node clustering, and node classification tasks.

Problem

• symmetric multi-type
• orthogonal non-negative matrix tri-factorization
• clustering
• network analysis
• non-negativity

Innovation

• orthogonal non-negative matrix tri-factorization
• multi-type matrix factorization
• fixed-point algorithm
• ADAM-based optimization
• graph embedding