sparseGeoHOPCA: A Geometric Solution to Sparse Higher-Order PCA Without Covariance Estimation

2025-06-10

AI Summary

This paper addresses key challenges in sparse high-order principal component analysis (SHOPCA) for high-dimensional tensors—namely, prohibitive covariance estimation cost, low efficiency of iterative deflation, and poor interpretability—by proposing a geometric tensor decomposition framework. Methodologically, it reformulates SHOPCA via mode-wise unfolding into a structured bilinear optimization problem and introduces geometric constraints to eliminate explicit covariance computation and iterative deflation entirely. Theoretically, we prove exact equivalence to the original SHOPCA formulation and derive a data-dependent error bound based on PCA residuals. The resulting algorithm achieves linear time complexity and natively handles imbalanced data. Experiments demonstrate: exact recovery of sparse supports on synthetic data; superior ImageNet image reconstruction quality; and stable classification accuracy under 10× compression. Our core contribution is the first geometric modeling perspective for SHOPCA, unifying computational efficiency, interpretability, and theoretical rigor.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Computer Vision: Other Foundations of Computer Vision
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

We propose sparseGeoHOPCA, a novel framework for sparse higher-order principal component analysis (SHOPCA) that introduces a geometric perspective to high-dimensional tensor decomposition. By unfolding the input tensor along each mode and reformulating the resulting subproblems as structured binary linear optimization problems, our method transforms the original nonconvex sparse objective into a tractable geometric form. This eliminates the need for explicit covariance estimation and iterative deflation, enabling significant gains in both computational efficiency and interpretability, particularly in high-dimensional and unbalanced data scenarios. We theoretically establish the equivalence between the geometric subproblems and the original SHOPCA formulation, and derive worst-case approximation error bounds based on classical PCA residuals, providing data-dependent performance guarantees. The proposed algorithm achieves a total computational complexity of $Oleft(sum_{n=1}^{N} (k_n^3 + J_n k_n^2) ight)$, which scales linearly with tensor size. Extensive experiments demonstrate that sparseGeoHOPCA accurately recovers sparse supports in synthetic settings, preserves classification performance under 10$ imes$ compression, and achieves high-quality image reconstruction on ImageNet, highlighting its robustness and versatility.

Problem

• Sparse higher-order PCA without covariance estimation
• Geometric solution for high-dimensional tensor decomposition
• Efficient sparse support recovery in unbalanced data

Innovation

• Geometric sparse tensor decomposition without covariance estimation
• Structured binary linear optimization for tractable subproblems
• Linear computational complexity scaling with tensor size