Spectrally-Corrected and Regularized QDA Classifier for Spiked Covariance Model

2025-03-17

AI Summary

To address the failure of classical Quadratic Discriminant Analysis (QDA) in high-dimensional settings—caused by spectral distortion of sample covariance matrices—this paper proposes Spectrally Regularized QDA (SR-QDA), a novel method integrating spectral correction with adaptive regularization. SR-QDA is the first to incorporate spectral correction into the QDA framework and selects the regularization parameter adaptively via maximization of the Fisher discriminant ratio. Under spiked covariance models, it achieves theoretically consistent and robust quadratic classification. Compared with standard QDA and regularized QDA (R-QDA), SR-QDA significantly improves classification accuracy in moderate-to-high dimensions. Its misclassification rate is proven to converge asymptotically, and extensive empirical evaluations confirm its stability and superior performance. The core innovation lies in the synergistic integration of spectral correction and Fisher-driven regularization, jointly ensuring statistical consistency and computational feasibility.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Mixed Discrete/Continuous Search
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Quadratic discriminant analysis (QDA) is a widely used method for classification problems, particularly preferable over Linear Discriminant Analysis (LDA) for heterogeneous data. However, QDA loses its effectiveness in high-dimensional settings, where the data dimension and sample size tend to infinity. To address this issue, we propose a novel QDA method utilizing spectral correction and regularization techniques, termed SR-QDA. The regularization parameters in our method are selected by maximizing the Fisher-discriminant ratio. We compare SR-QDA with QDA, regularized quadratic discriminant analysis (R-QDA), and several other competitors. The results indicate that SR-QDA performs exceptionally well, especially in moderate and high-dimensional situations. Empirical experiments across diverse datasets further support this conclusion.

Problem

• Addresses QDA's ineffectiveness in high-dimensional settings
• Proposes SR-QDA with spectral correction and regularization
• Improves classification performance in moderate to high dimensions

Innovation

• Spectral correction enhances QDA performance
• Regularization optimizes Fisher-discriminant ratio
• SR-QDA excels in high-dimensional data