🤖 AI Summary
This work proposes RRLDA-RK, a fast, parameter-free iterative algorithm for reduced-rank linear discriminant analysis (RRLDA) that operates effectively in both classical and high-dimensional settings without relying on strong assumptions or explicit regularization tuning. By integrating techniques from high-dimensional statistics and numerical linear algebra, the method inherently possesses implicit regularization properties and automatically converges to the minimum-norm solution. This ensures theoretical rigor while substantially improving computational efficiency. Empirical evaluations on real high-dimensional datasets demonstrate that RRLDA-RK achieves excellent classification performance alongside strong stability and scalability, addressing the high computational cost typically associated with traditional RRLDA approaches in large-scale, high-dimensional scenarios.
📝 Abstract
Reduced-rank linear discriminant analysis (RRLDA) is a foundational method of dimension reduction for classification that has been useful in a wide range of applications. The goal is to identify an optimal subspace to project the observations onto that simultaneously maximizes between-group variation while minimizing within-group differences. The solution is straight forward when the number of observations is greater than the number of features but computational difficulties arise in both the high-dimensional setting, where there are more features than there are observations, and when the data are very large. Many works have proposed solutions for the high-dimensional setting and frequently involve additional assumptions or tuning parameters. We propose a fast and simple iterative algorithm for both classical and high-dimensional RRLDA on large data that is free from these additional requirements and that comes with guarantees. We also explain how RRLDA-RK provides implicit regularization towards the least norm solution without explicitly incorporating penalties. We demonstrate our algorithm on real data and highlight some results.