Infinitesimal Higher-Order Spectral Variations in Rectangular Real Random Matrices

2025-06-04

AI Summary

Closed-form expressions for higher-order Fréchet derivatives of singular values of real rectangular matrices have long been unavailable. Method: This paper establishes a unified analytical framework based on Kato’s analytic perturbation theory: a rectangular matrix is embedded into a self-adjoint block operator; using reduced resolvents and Kronecker product representations, it rigorously characterizes the *n*-th-order spectral variation under asymmetric infinitesimal perturbations in finite-dimensional Hilbert spaces. Contribution/Results: We derive the first general closed-form formula for the *n*-th-order Fréchet derivative of singular values; in particular, we fully obtain the long-missing Hessian matrix (*n* = 2) of singular values. The framework yields a computationally tractable tool for higher-order spectral sensitivity analysis of random matrices and finds direct applications in modeling adversarial robustness in deep learning.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

We present a theoretical framework for deriving the general $n$-th order Fr'echet derivatives of singular values in real rectangular matrices, by leveraging reduced resolvent operators from Kato's analytic perturbation theory for self-adjoint operators. Deriving closed-form expressions for higher-order derivatives of singular values is notoriously challenging through standard matrix-analysis techniques. To overcome this, we treat a real rectangular matrix as a compact operator on a finite-dimensional Hilbert space, and embed the rectangular matrix into a block self-adjoint operator so that non-symmetric perturbations are captured. Applying Kato's asymptotic eigenvalue expansion to this construction, we obtain a general, closed-form expression for the infinitesimal $n$-th order spectral variations. Specializing to $n=2$ and deploying on a Kronecker-product representation with matrix convention yield the Hessian of a singular value, not found in literature. By bridging abstract operator-theoretic perturbation theory with matrices, our framework equips researchers with a practical toolkit for higher-order spectral sensitivity studies in random matrix applications (e.g., adversarial perturbation in deep learning).

Problem

• Deriving n-th order Fréchet derivatives of singular values in real rectangular matrices
• Overcoming challenges in closed-form expressions for higher-order spectral variations
• Providing a practical toolkit for spectral sensitivity studies in random matrices

Innovation

• Using Kato's perturbation theory for derivatives
• Embedding matrices into self-adjoint operators
• Closed-form Hessian via Kronecker products