🤖 AI Summary
Traditional proofs of Singular Value Decomposition (SVD) rely on the spectral theorem, lacking geometric intuition and failing to reveal deep connections with machine learning algorithms. This work reconstructs SVD from an ellipsoidal geometry perspective, transforming the recursive process of maximizing stretch into an algorithmic mechanism. By integrating geometric constructions, gradient descent stopping criteria, and duality analysis of kernel methods, it achieves a "proof-as-algorithm" paradigm that derives SVD without presupposing the spectral theorem. Furthermore, this study establishes explicit mappings between SVD and core algorithms such as Principal Component Analysis (PCA) and PageRank, thereby unifying the foundational logic of linear algebra and machine learning. Finally, it provides a pedagogical framework amenable to manual verification alongside a discussion of theoretical boundaries.
📝 Abstract
This article is a geometric rediscovery of the singular value decomposition, with a further claim: the construction it builds is the machinery behind much of machine learning. The same argument that answers an idle question about ellipses is the algorithm behind principal component analysis, kernel methods, and PageRank, and it is not only the results that transfer but the proofs themselves, run as procedures.
The usual introduction states $A = UΣV^T$ and justifies it via the spectral theorem applied to $A^T A$. This is correct but unilluminating, since it assumes a powerful theorem to reach a result that is, in the end, about ellipses. Part I reverses the order. A linear map sends the unit circle to an ellipse; one asks which input directions map to its axes, and finds, example after example, that they are perpendicular. In the plane this can be watched: rotate a frame, track how far its images are from perpendicular, and a sign change forces a frame where they are exactly perpendicular, which is also where the map stretches hardest. Maximizing the stretch and recursing generalizes this to n dimensions, with singular values falling out in order, and the construction proves the spectral theorem rather than assuming it.
Part II puts each construction to work: maximize-and-recurse becomes the power method and PageRank; the lemma locating the maximizer becomes the stopping rule of gradient descent; the duality between $A^T A$ and $A A^T$ becomes the transport at the heart of kernel PCA. Each connection is stated with its boundary, saying what the decomposition supplies and where another idea takes over. Prerequisites are the standard sophomore sequence, and the worked examples are small enough to check by hand.