🤖 AI Summary
This study addresses the numerical instability in extreme learning machine (ELM) training caused by ill-conditioned hidden-layer matrices during pseudoinverse computation. From a spectral perspective, it reveals how perturbations in output weights are amplified by the smallest singular value and quantifies instability via the condition number. The work establishes, for the first time, a systematic theoretical link between ELM numerical stability and the singular value structure of the hidden-layer matrix, proposing a spectral-based stability criterion. Leveraging singular value decomposition (SVD) and iterative hyperpower methods to compute the pseudoinverse, combined with random feature theory, it analyzes how network width influences the condition number. Experiments demonstrate that SVD is the most robust under ill-conditioned scenarios, whereas iterative methods exhibit greater sensitivity to spectral properties, confirming that stability is predominantly governed by the singular value spectrum.
📝 Abstract
Extreme Learning Machine (ELM) computes output weights analytically using the Moore-Penrose pseudoinverse. Although this leads to fast training, its numerical stability depends strongly on the conditioning of the hidden layer matrix. This paper studies pseudoinverse-based ELM from a spectral perspective. We show that the smallest singular value governs perturbation amplification in the output weights, while the condition number provides a quantitative measure of hidden-layer instability. We compare SVD-based pseudoinverse computation with iterative hyperpower methods and discuss width-dependent conditioning through a random feature interpretation. Experiments on synthetic matrices and ELM benchmarks show that SVD-based methods remain the most reliable under ill conditioning, while iterative methods are more sensitive to spectral properties. The results suggest that ELM stability is fundamentally governed by the singular value structure of the hidden layer matrix.