Subspace Ordering for Maximum Response Preservation in Sufficient Dimension Reduction

2025-10-31

AI Summary

Traditional sufficient dimension reduction (SDR) relies on eigenvalue-based subspace ordering, yet eigenvalues do not necessarily reflect predictive relevance, limiting post-reduction prediction performance. To address this, we propose a response-prediction-oriented subspace reordering framework that—uniquely among SDR methods—abandons eigenvalue criteria entirely. Instead, we uniformly adopt the absolute value of the independent *t*-statistic (for classification) and the *F*-statistic (for regression) as importance measures for direction vectors, applied after generalized eigendecomposition. We establish theoretical consistency and asymptotic optimality of the resulting estimator. Extensive experiments across binary classification, multiclass classification, and regression tasks demonstrate that our criterion significantly improves both predictive accuracy and subspace estimation fidelity, consistently outperforming multiple state-of-the-art SDR approaches.

Technology Category

• Machine Learning: Dimensionality Reduction/Feature Selection
• Intelligent Robots: State Estimation
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Sufficient dimension reduction (SDR) methods aim to identify a dimension reduction subspace (DRS) that preserves all the information about the conditional distribution of a response given its predictor. Traditional SDR methods determine the DRS by solving a method-specific generalized eigenvalue problem and selecting the eigenvectors corresponding to the largest eigenvalues. In this article, we argue against the long-standing convention of using eigenvalues as the measure of subspace importance and propose alternative ordering criteria that directly assess the predictive relevance of each subspace. For a binary response, we introduce a subspace ordering criterion based on the absolute value of the independent Student's t-statistic. Theoretically, our criterion identifies subspaces that achieve the local minimum Bayes' error rate and yields consistent ordering of directions under mild regularity conditions. Additionally, we employ an F-statistic to provide a framework that unifies categorical and continuous responses under a single subspace criterion. We evaluate our proposed criteria within multiple SDR methods through extensive simulation studies and applications to real data. Our empirical results demonstrate the efficacy of reordering subspaces using our proposed criteria, which generally improves classification accuracy and subspace estimation compared to ordering by eigenvalues.

Problem

• Challenges eigenvalue-based subspace importance in dimension reduction
• Proposes predictive relevance criteria for subspace ordering
• Unifies categorical and continuous response frameworks in SDR

Innovation

• Replaces eigenvalue ordering with predictive relevance criteria
• Uses t-statistic for binary response subspace ordering
• Unifies categorical and continuous responses via F-statistic framework