🤖 AI Summary
Traditional spectral embedding (SE) methods suffer from three key limitations: poor generalizability to out-of-sample nodes, computational intractability on large-scale graphs, and insufficient separability of learned eigenvectors—hindering downstream clustering and visualization. To address these, we propose GrEASE, the first end-to-end differentiable and generalizable deep spectral embedding framework. Its core contributions are: (1) a differentiable neural architecture approximating the graph Laplacian operator, enabling generalizable SE learning; (2) a feature-decoupling regularization mechanism that explicitly enhances geometric separability of embeddings; and (3) NUMAP—the first generalizable variant of UMAP. Experiments demonstrate that GrEASE consistently approximates ground-truth spectral embeddings across diverse datasets, enables real-time out-of-sample embedding, and significantly improves both the generalizability and efficiency of UMAP. The implementation is publicly available.
📝 Abstract
Spectral Embedding (SE) is a popular method for dimensionality reduction, applicable across diverse domains. Nevertheless, its current implementations face three prominent drawbacks which curtail its broader applicability: generalizability (i.e., out-of-sample extension), scalability, and eigenvectors separation. In this paper, we introduce GrEASE: Generalizable and Efficient Approximate Spectral Embedding, a novel deep-learning approach designed to address these limitations. GrEASE incorporates an efficient post-processing step to achieve eigenvectors separation, while ensuring both generalizability and scalability, allowing for the computation of the Laplacian's eigenvectors on unseen data. This method expands the applicability of SE to a wider range of tasks and can enhance its performance in existing applications. We empirically demonstrate GrEASE's ability to consistently approximate and generalize SE, while ensuring scalability. Additionally, we show how GrEASE can be leveraged to enhance existing methods. Specifically, we focus on UMAP, a leading visualization technique, and introduce NUMAP, a generalizable version of UMAP powered by GrEASE. Our codes are publicly available.