principal component analysis

To address the “curse of dimensionality” arising from high-dimensional design spaces in functional surface design, this study systematically reviews and innovates dimensionality reduction (DR) methods for shape optimization. Adopting the scoping review methodology—novel in engineering design—we establish a unified taxonomy encompassing linear techniques (PCA, kernel PCA), nonlinear approaches (t-SNE, autoencoders, VAEs), and physics-informed methods (physics-constrained embedding networks, surrogate-model-coupled frameworks). We propose a novel physics-informed embedding framework that significantly enhances interpretability and engineering applicability of DR results. Experimental evaluation demonstrates that optimization efficiency improves by 3–10× after DR, design space explorability increases, and physical consistency of optimal solutions is markedly improved—validating both computational efficacy and domain fidelity.

This work addresses the challenges in nonlinear dimensionality reduction of simultaneously preserving global and local structures and providing interpretability during embedding. We propose a novel embedding method that integrates spectral decomposition with multiscale cross-entropy optimization. By analyzing the influence of spectral modes on embeddings from a graph frequency-domain perspective, our approach uniquely combines spectral basis representations with multiscale nonlinear dimensionality reduction. This integration enables a balanced preservation of both global and local manifold structures while maintaining continuity. To enhance interpretability, we introduce glyph-augmented scatterplots for visual exploration of the embedding process. Quantitative evaluations and case studies demonstrate the effectiveness of our method in achieving structurally faithful and interpretable low-dimensional representations.

This work proposes Functional Tucker Decomposition (FTD) to address the limitation of traditional tensor decomposition, which loses essential continuity structures when discretizing multidimensional data generated by continuous processes. By embedding Reproducing Kernel Hilbert Spaces (RKHS) into the Tucker framework, FTD adaptively learns expressive factors through mode-wise continuity constraints, enabling the modeling of continuous patterns without requiring predefined basis functions while preserving the multilinear subspace structure. The method uniquely unifies structural fidelity with functional flexibility. Experimental results on hyperspectral image classification and multivariate time series analysis demonstrate that FTD significantly outperforms existing approaches, validating the effectiveness of integrating continuous function modeling with tensor decomposition.

Diffusion maps are widely used in nonlinear manifold learning, yet their performance critically depends on data preprocessing, parameter selection, and—most notably—the choice of diffusion components. Current practices often default to using the leading components associated with the largest eigenvalues, but lack systematic guidance for this selection. This work provides a comprehensive review of diffusion map methodology and integrates recent advances in component importance assessment. Through empirical analysis, we demonstrate that the most informative low-dimensional components do not necessarily correspond to the largest eigenvalues, thereby challenging conventional assumptions. The study advocates for data-driven strategies to identify relevant components, which substantially enhances the interpretability and practical utility of dimensionality reduction results.

Learning the spectral decomposition of the Laplacian operator from unstructured high-dimensional data—such as 3D point clouds or image manifolds—typically requires explicit discretization, mesh construction, or solving eigenvalue problems, limiting scalability and applicability to unknown geometries. Method: We propose an end-to-end framework that jointly learns the implicit spectral basis, eigenvalues, and density-induced metric of the Laplacian directly from raw data—without constructing differential operators, discretizing domains, or solving eigenproblems. Leveraging optimal approximation theory, we parameterize the operator and its spectrum via neural networks, minimizing reconstruction error under a probe function distribution. Contribution/Results: The method is fully unsupervised, mesh-free, dimension-agnostic, and geometry-agnostic. Experiments demonstrate that the learned spectral basis exhibits Laplacian-like properties—yielding interpretable, generalizable, and scalable representations across diverse unstructured datasets, including high-dimensional settings.

We address regression with tensor-valued predictors. We propose two nonlinear sufficient dimension reduction (SDR) methods—based respectively on Tucker and CP decompositions—that preserve both the intrinsic multimodal structure and semantic interpretability of tensor modes. Our key contribution is the first formulation of nonlinear SDR that jointly maintains the original tensor structure and mode-wise interpretability. Theoretically, we establish Fisher consistency, statistical consistency, and convergence rate guarantees. Algorithmically, our approach integrates least-squares sequential optimization with singular value decomposition to balance geometric fidelity and model parsimony. Extensive simulations and two real-data experiments demonstrate that our methods reduce parameter count by approximately one order of magnitude, while achieving significantly higher estimation accuracy and predictive performance than state-of-the-art alternatives. Moreover, they deliver strong interpretability and computational efficiency.

To address the limited reconstruction accuracy of conventional linear dimensionality reduction methods, this paper proposes a nonlinear-enhanced manifold learning framework grounded in reproducing kernel Hilbert spaces (RKHS). The method extends generic linear dimensionality reduction by incorporating a learnable, optimal nonlinear correction term, which is uniformly modeled via kernel methods to capture arbitrary nonlinear structures—including polynomial and RBF kernels—and jointly optimized with the low-dimensional embedding under a least-squares criterion. Theoretically, the correction error is guaranteed to decrease monotonically with increasing latent dimensionality. Experimental results on multiple benchmark datasets demonstrate that the proposed approach significantly outperforms proper orthogonal decomposition (POD) and state-of-the-art quadratic manifold methods, achieving both superior approximation accuracy and reduced training cost.

Hyperspectral imaging (HSI) suffers from high spectral dimensionality and substantial redundancy, necessitating dimensionality reduction methods that jointly optimize predictive performance and interpretability. This paper proposes a post-hoc interpretable band selection framework for HSI: it introduces the deletion-insertion evaluation paradigm to HSI band selection for the first time, quantifying each band’s influence score on a pre-trained classifier’s decision. High-impact bands with clear physical meaning are then selected. The method integrates model-aligned contribution analysis, influence score aggregation, and empirical validation. Experiments on the Pavia University and Salinas datasets demonstrate that classifiers using only 30 selected bands achieve accuracy comparable to or exceeding that of full-spectrum models, while significantly reducing computational cost. Crucially, the approach ensures both interpretability—through transparent, physics-grounded band selection—and generalizability across diverse scenes and classifiers.

To address the limited interpretability in unsupervised dimensionality reduction of unlabeled network data, this paper proposes Centroid Subspace Analysis (CSA). CSA abandons conventional vector-based linear subspace modeling and instead treats point sets—specifically, cospectral network equivalence classes—as fundamental units, constructing a centroid subspace on the Riemannian manifold to achieve nonlinear dimensionality reduction. We further design a cospectral-equivalence-class–driven network embedding strategy that maps original networks onto well-structured point sets residing on a geometric manifold. Experiments on both synthetic and real-world network datasets demonstrate that CSA significantly outperforms tangent Principal Component Analysis (tPCA), achieving superior dimensionality reduction fidelity and enhanced semantic interpretability. By unifying spectral graph theory, Riemannian geometry, and unsupervised representation learning, CSA establishes a novel geometric paradigm for unsupervised network representation learning.

Traditional sufficient dimension reduction (SDR) relies on eigenvalue-based subspace ordering, yet eigenvalues do not necessarily reflect predictive relevance, limiting post-reduction prediction performance. To address this, we propose a response-prediction-oriented subspace reordering framework that—uniquely among SDR methods—abandons eigenvalue criteria entirely. Instead, we uniformly adopt the absolute value of the independent *t*-statistic (for classification) and the *F*-statistic (for regression) as importance measures for direction vectors, applied after generalized eigendecomposition. We establish theoretical consistency and asymptotic optimality of the resulting estimator. Extensive experiments across binary classification, multiclass classification, and regression tasks demonstrate that our criterion significantly improves both predictive accuracy and subspace estimation fidelity, consistently outperforming multiple state-of-the-art SDR approaches.

Traditional dimensionality reduction methods face a fundamental trade-off: linear techniques such as PCA fail to capture nonlinear manifold structures, whereas nonlinear approaches—including autoencoders and graph-based embeddings—often lack interpretability or induce geometric distortion. This paper proposes the Variational Manifold Embedding (VME) framework, which formulates dimensionality reduction as an optimal manifold embedding problem constrained by partial differential equations (PDEs). VME jointly achieves strong nonlinear expressivity and rigorous mathematical interpretability. By incorporating symmetry constraints and leveraging the variational principle on manifolds, VME enables analytical characterization of embedding solutions and strictly reduces to PCA under specific conditions. Crucially, VME avoids the manifold distortion inherent in graph-based methods, yields differentiable and verifiable embeddings, and supports formal theoretical analysis of embedding properties—including stability, uniqueness, and geometric fidelity.