Asymptotic behavior of eigenvalues of large rank perturbations of large random matrices

2025-07-16

AI Summary

This work investigates the asymptotic eigenvalue behavior of deformed Wigner random matrices under high-rank perturbations, addressing the limitation of existing theory—restricted to finite-rank perturbations—to better model the spectral structure of trained deep neural network (DNN) weight matrices, which decompose as $W = R + S$, where $R$ is a random component and $S$ is a highly correlated, low-rank (but rank-growing) signal component. We develop a novel analytical framework integrating random matrix theory, spectral analysis, and high-dimensional probability. For the first time, we extend asymptotic spectral analysis to the regime where the rank of $S$ grows with the matrix dimension. Our results provide precise characterizations of both the limiting spectral distribution and the locations of outlier eigenvalues under high-rank deformation. These theoretical advances establish a foundational mathematical basis for understanding the spectral properties of DNN weight matrices and for designing theoretically grounded, interpretable pruning algorithms.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Search and Optimization: Non-convex Optimization
• Computer Vision: Other Foundations of Computer Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

The paper is concerned with deformed Wigner random matrices. These matrices are closely connected with Deep Neural Networks (DNNs): weight matrices of trained DNNs could be represented in the form $R + S$, where $R$ is random and $S$ is highly correlated. The spectrum of such matrices plays a key role in rigorous underpinning of the novel pruning technique based on Random Matrix Theory. Mathematics has been done only for finite-rank matrix $S$. However, in practice rank may grow. In this paper we develop asymptotic analysis for the case of growing rank.

Problem

• Analyze eigenvalues of large perturbed random matrices
• Study spectrum of deformed Wigner matrices in DNNs
• Extend asymptotic analysis to growing-rank perturbations

Innovation

• Deformed Wigner matrices for DNN weight analysis
• Asymptotic analysis of growing-rank perturbations
• Spectrum study enables RMT-based pruning