Riemannian Optimization for Hadamard Products of Low-Rank Matrices

2026-05-31

AI Summary

This work addresses the optimization challenges in low-rank matrix Hadamard product modeling, which arise from row- and column-scaling symmetries. The authors formulate the problem as an optimization over a Riemannian quotient manifold and introduce a block-diagonal Riemannian metric derived from the pullback of the Frobenius inner product. This metric is invariant under the full symmetry group of scaling transformations. They further devise a parameter-free Gauss–Newton step-size strategy that leverages this geometric structure. The resulting algorithm achieves linear computational complexity with respect to the number of observed entries and demonstrates both efficiency and effectiveness on synthetic and real-world datasets.

Abstract

The elementwise Hadamard product of two low-rank matrices provides a parameter-efficient model for data with multiplicative structure, but its modeling is challenging due to the presence of additional symmetries under coupled row/column scalings between the two factors. In order to leverage the geometry of the space, we formulate the learning of such matrices as optimization on a Riemannian quotient manifold. We propose a novel block-diagonal Riemannian metric derived from the pullback of the Frobenius inner product. The metric is shown to be invariant under the full symmetry group. We develop a Riemannian gradient descent algorithm that uses a tuning-free Gauss--Newton step size and scales linearly in the number of observed entries per iteration. Experiments on real and synthetic datasets illustrate the efficacy of our proposed Riemannian approach.

Problem

• Hadamard product
• low-rank matrices
• symmetry
• Riemannian optimization
• matrix factorization

Innovation

• Riemannian optimization
• Hadamard product
• low-rank matrices
• quotient manifold
• symmetry-invariant metric