Low-Rank Dependence Decomposition via Accelerated Symmetric Non-negative Matrix Factorization

2026-07-27

AI Summary

This work addresses the memory and computational bottlenecks in low-rank decomposition of million-dimensional dependency matrices—common in financial risk factor estimation—by proposing a symmetric nonnegative matrix factorization (SymNMF) framework based on trace identities that avoids explicitly forming dense intermediate matrices. Integrating multi-GPU and distributed strategies, the study introduces three novel algorithms: Piecewise AdaGrad, Row-Stochastic SVRG, and Block-SVRG AdaptGrow, and presents the first systematic evaluation of optimizer convergence and efficiency in ultra-large-scale SymNMF. Experiments demonstrate that a single GPU handles problems with $n \approx 10^5$, while multi-node deployments scale beyond $n > 10^6$. Full-batch AdaGrad achieves the fastest convergence on correlation matrices, whereas Block-SVRG AdaptGrow excels on tail-dependence matrices, further confirming the necessity of soft clustering when factors degenerate to single components.

Abstract

Symmetric non-negative matrix factorization (SymNMF) recovers latent group structure from a dependence matrix, but its dense, quadratic-memory objective has confined prior work to moderate sizes. We present a large-scale GPU study of seven algorithm families (over 30 configurations) on absolute Pearson correlation and tail pairwise dependence matrices from Extreme Value Theory, two proxies for empirical risk-factor estimation on large portfolios. A trace-identity reformulation eliminates all $n \times n$ intermediates, so a single GPU reaches $n \approx 10^5$ and multi-node distribution scales to $n = 10^6$ and beyond. Under a two-phase protocol, eleven methods converge at moderate scale; six remain efficient enough at $n = 10^5$ (five AdaGrad-family plus ADMM), and five AdaGrad-family methods still converge at $n = 10^6$: AdaGrad, RMSprop, and three we introduce (Piecewise AdaGrad, Row-Stochastic SVRG, Block-SVRG AdaptGrow). At $n = 10^6$ the fastest solver tracks the matrix spectrum: Block-SVRG AdaptGrow wins on the flat, ill-conditioned tail-dependence spectrum, where its lower per-iteration cost decides a long factorization, and full-batch AdaGrad wins on the dominant-low-rank correlation spectrum, where the run is short. We also benchmark spherical K-means as a hard-label baseline: cheaper when angular cluster structure is present, yet provably degenerate once the matrix collapses toward a single common factor, where the soft factorization remains necessary.

Problem

• Symmetric Non-negative Matrix Factorization
• Low-Rank Dependence Decomposition
• Large-Scale Optimization
• Tail Pairwise Dependence
• Empirical Risk-Factor Estimation

Innovation

• Symmetric Non-negative Matrix Factorization
• Trace-Identity Reformulation
• Large-Scale GPU Acceleration
• Tail Dependence
• AdaGrad-family Optimizers