Implicit Regularization in Perturbed Deep Matrix Factorization: Spectral Conditions and Stability

2026-05-27

AI Summary

This study investigates the stability of implicit low-rank regularization in deep matrix factorization under noisy perturbations. By integrating spectral analysis, gradient flow dynamics, and perturbation theory, it characterizes how the spectral structure of the target matrix, initialization, and step size influence the existence of a low-rank phase. The work establishes, for the first time, explicit spectral conditions under which implicit low-rank regularization persists in the presence of perturbations. Theoretically, it proves that the optimization algorithm still converges to a low-rank solution despite noise, and derives explicit bounds on both iteration complexity and eigenvalue recovery error. These results quantitatively reveal how perturbation magnitude affects stability. Numerical experiments corroborate the theoretical findings.

Abstract

This paper studies the stability of low-rank implicit regularization in perturbed deep matrix factorization, where the target matrix is corrupted by a noise matrix. We first derive sufficient spectral conditions under which gradient descent exhibits a low-rank phase in the noiseless setting. These conditions show how the target spectrum, initialization, and step size jointly determine the existence of a nonempty low-rank interval. We then analyze the perturbed gradient descent dynamics, proving convergence guarantees and quantifying how the perturbation affects iteration complexity and eigenvalue recovery. Finally, we show that the low-rank phase persists under perturbation, with explicit dependence on the perturbation size. Numerical experiments support the theoretical findings.

Problem

• implicit regularization
• deep matrix factorization
• low-rank
• perturbation
• stability

Innovation

• implicit regularization
• deep matrix factorization
• spectral conditions
• perturbation stability
• gradient descent dynamics