compute sparse principal components

Sparse PCA faces two key challenges in high-dimensional, low-sample-size settings: (1) excessive regularization biases singular vectors away from the true underlying structure, and (2) non-orthogonality among components induces information redundancy, compromising explained-variance estimation. To address these, we propose *inherently sparse PCA*, a regularization-free approach that leverages the data’s intrinsic sparse block-diagonal covariance structure. Instead of imposing sparsity via ℓ₁ penalties, our method identifies and isolates coherent submatrices to directly yield naturally sparse and strictly orthogonal principal components. This design ensures structural fidelity and interpretability from the outset, circumventing distortions caused by over-sparsification and post-hoc orthogonalization. Experiments on synthetic and real-world high-dimensional datasets demonstrate substantial improvements in component stability and accuracy of explained-variance estimation. Our work establishes a new paradigm for interpretable dimensionality reduction grounded in structural priors rather than ad hoc regularization.

This work addresses the long-standing challenge in high-dimensional sparse principal component analysis (SPCA) of simultaneously ensuring orthogonality and statistical optimality across multiple components. Unlike conventional sequential extraction and deflation strategies—which inherently violate orthogonality—we propose the first unified optimization framework for jointly learning multiple sparse principal components. Our method explicitly enforces orthogonality via a rank constraint on the component matrix and jointly optimizes sparsity and low-rank structure. Key technical contributions include: (i) the first rank-constrained orthogonal modeling formulation for SPCA; (ii) a tight semidefinite relaxation enhanced with second-order cone constraints; (iii) a novel combinatorial upper bound on explained variance derived via support-set enumeration; and (iv) a certifiably near-optimal solver achieving provable optimality gaps of 0%–15%. Experiments on real-world datasets with $p = 100$–$1000$ features and $r = 2$–$3$ components yield strictly orthogonal, sparse loadings while matching or exceeding state-of-the-art variance explanation—eliminating orthogonality violations entirely.

This study addresses the limitations of traditional sparse principal component analysis (SPCA) in high-dimensional settings, where uniform penalization across all variables undermines interpretability and stability by failing to differentiate variable importance. To overcome this, the authors propose SP-SPCA, a novel approach that introduces a single adaptive balancing parameter within an L2-regularized framework to differentially modulate penalty strengths across variables, thereby flexibly trading off sparsity against explained variance. Coupled with an efficient optimization algorithm, SP-SPCA maintains computational efficiency while significantly enhancing feature selection accuracy, model stability, and result interpretability. Experiments on both synthetic and real-world datasets—including crime and financial market data—demonstrate that SP-SPCA more accurately recovers sparse loading structures, effectively eliminates noise variables, and retains higher cumulative variance with fewer selected variables, outperforming existing SPCA methods.

This study addresses the challenge that traditional sparse principal component analysis (PCA) struggles to simultaneously achieve high explained variance, sparsity, and non-redundancy among multiple principal components in high-dimensional data. To overcome this limitation, the authors propose msPCA, a novel method based on an alternating maximization algorithm that enforces two forms of non-redundancy constraints—either orthogonality of loadings or zero correlation among principal components—while preserving high variance explanation and sparsity. The accompanying open-source R package efficiently scales to datasets with thousands of features, demonstrating superior performance over existing approaches by striking an effective balance between controllable sparsity and computational efficiency.

Existing sparse principal component analysis (SPCA) methods struggle to simultaneously achieve sparsity, orthogonality, and global optimality. This work proposes the GS-SPCA algorithm, which, for the first time, delivers a certifiably globally optimal solution under strict ℓ₀ sparsity constraints while guaranteeing orthogonality among principal components. The method integrates Gram–Schmidt orthogonalization, branch-and-bound optimization, and a threshold-based block partitioning with block-diagonal approximation of the covariance matrix to construct an efficient decomposition framework. Furthermore, it introduces an ε-optimal acceleration strategy that substantially enhances computational efficiency in multi-component settings while maintaining controllable solution accuracy.

Traditional sparse principal component analysis (SPCA) struggles to simultaneously achieve sparsity, uncorrelated components, and high explained variance. This work proposes a least-squares sparse principal component analysis (LS-SPCA) framework that optimizes variable selection strategies to yield sparse, weakly correlated principal components while maintaining high correlation with conventional PCs. LS-SPCA is the first method to jointly control sparsity, explained variance, and inter-component correlation, and it accommodates diverse variable selection algorithms—including forward selection, stepwise forward selection, and backward elimination. Implemented with an efficient C++ matrix engine optimized for both tall-and-skinny and short-and-wide matrices and interfaced through R, LS-SPCA demonstrates superior performance on real high-dimensional datasets, significantly outperforming existing approaches in terms of explained variance, sparsity, and computational efficiency.

This work addresses the high computational complexity of Highly Adaptive Lasso (HAL) in high-dimensional nonparametric regression by proposing a dimensionality reduction approach based on principal component analysis (PCA), yielding the PCHAL and PCHAR estimators. The method substantially reduces computational cost without relying on the outcome variable, while maintaining empirical performance comparable to that of the original HAL and HAR estimators. Theoretical analysis reveals a spectral connection between the principal components of the HAL/HAR Gram operator and the discrete sine basis, uncovering an intrinsic Fourier-type structure. This insight provides both a novel perspective and practical tools for efficient nonparametric estimation in high-dimensional settings.

This work proposes a novel method—nuclear-norm-penalized principal covariates regression (PcovR-nnp)—to address the challenges of high-dimensional regression, where dimensionality reduction and regularized coefficient estimation are typically performed sequentially in an ad hoc order, compromising model stability. By introducing the nuclear norm into the principal covariates regression framework for the first time, PcovR-nnp jointly optimizes matrix decomposition and regularized regression, thereby simultaneously achieving dimension selection and coefficient estimation. This integrated approach eliminates the need for subjective, stepwise procedural choices inherent in conventional pipelines and substantially enhances both the accuracy and robustness of high-dimensional regression modeling.

This work addresses the challenge of preserving data variance while avoiding redundancy in unsupervised feature selection by proposing a greedy selection method grounded in a weighted PCA loading space. The approach incorporates a null-space ablation mechanism that, after each selection step, removes the variance direction of already-selected features, thereby enforcing subsequent selections to cover orthogonal subspaces within the covariance structure. The method establishes a theoretical connection to monotone submodular maximization and introduces a reproducible hyperparameter selection criterion based on sensitivity scanning of reconstruction mean squared error (MSE). Experimental results across eight benchmark datasets demonstrate that the proposed method consistently achieves the lowest reconstruction MSE, offers speedups of 10–140× over graph-based baselines, and reduces MSE by 10%–73% compared to standard PCA without ablation at equivalent dimensions.