Graph Regularized PCA

2026-01-15

AI Summary

This work proposes Graph-Regularized PCA (GR-PCA) to address the limitation of traditional PCA, which assumes isotropic noise and struggles with high-dimensional data exhibiting inter-feature dependencies—i.e., non-spherical noise covariance—leading to principal components lacking structural interpretability. GR-PCA is the first to incorporate a graph Laplacian regularizer into the PCA framework, modeling conditional dependencies among variables via a sparse precision graph and guiding loadings learning through low-frequency graph Fourier modes. This approach suppresses high-frequency noise while preserving graph-consistent signals. Experiments demonstrate that GR-PCA significantly enhances structural fidelity and interpretability across diverse synthetic and real-world settings, without compromising predictive performance. It achieves more concentrated variance capture over target support sets, lower graph energy in loadings, and exhibits strong generalization and scalability.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models
• Computer Vision: Other Foundations of Computer Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

High-dimensional data often exhibit dependencies among variables that violate the isotropic-noise assumption under which principal component analysis (PCA) is optimal. For cases where the noise is not independent and identically distributed across features (i.e., the covariance is not spherical) we introduce Graph Regularized PCA (GR-PCA). It is a graph-based regularization of PCA that incorporates the dependency structure of the data features by learning a sparse precision graph and biasing loadings toward the low-frequency Fourier modes of the corresponding graph Laplacian. Consequently, high-frequency signals are suppressed, while graph-coherent low-frequency ones are preserved, yielding interpretable principal components aligned with conditional relationships. We evaluate GR-PCA on synthetic data spanning diverse graph topologies, signal-to-noise ratios, and sparsity levels. Compared to mainstream alternatives, it concentrates variance on the intended support, produces loadings with lower graph-Laplacian energy, and remains competitive in out-of-sample reconstruction. When high-frequency signals are present, the graph Laplacian penalty prevents overfitting, reducing the reconstruction accuracy but improving structural fidelity. The advantage over PCA is most pronounced when high-frequency signals are graph-correlated, whereas PCA remains competitive when such signals are nearly rotationally invariant. The procedure is simple to implement, modular with respect to the precision estimator, and scalable, providing a practical route to structure-aware dimensionality reduction that improves structural fidelity without sacrificing predictive performance.

Problem

• high-dimensional data
• non-isotropic noise
• feature dependencies
• graph structure
• dimensionality reduction

Innovation

• Graph Regularized PCA
• graph Laplacian
• structured dimensionality reduction
• sparse precision graph
• low-frequency Fourier modes