Streaming PCA: averaging from a geometric perspective

2026-09-19

AI Summary

本文研究了在内存限制下，利用Oja算法从几何视角解决PCA问题，开发了一种无需预先了解特征间隙的自适应平均方法以优化收敛速度。

Abstract

We study principal component analysis (PCA) under memory constraints, a setting that is increasingly important in large-scale data analysis. Our focus is on Oja's algorithm, which is a one-pass, memory-efficient algorithm requiring only $O(p)$ storage in the rank-one case and $O(pk)$ storage for $k$- PCA. The main goal is to develop a procedure based on Oja's algorithm that is optimal without prior knowledge of the eigengap, which is otherwise needed to tune the learning rates effectively. We do this by first introducing a geometric perspective on the convergence of $k$-PCA: by embedding the Oja iterates into the exterior space, we establish an exact equivalence between $k$-PCA in the ambient space and $1$-PCA in the exterior space. This geometric viewpoint enables us to establish several convergence guarantees, including one based only on a single learning-rate schedule. Building on this geometric perspective, we develop an averaging theory for $k$-PCA and show that the resulting averaged estimator is adaptive: it achieves the nearly optimal convergence rate without prior knowledge of the eigengap. As an application, we construct memory-efficient estimators for elliptical component analysis (ECA) \cite{han2014scale,han2018eca}. Simulation studies and real data analysis are conducted to demonstrate the benefits of our proposed algorithms.

Problem

• Principal Component Analysis
• Memory Constraints
• Oja's Algorithm
• Eigengap

Innovation

• geometric perspective
• Oja's algorithm
• adaptive convergence rate
• streaming PCA