subspace-aligned rewiring

Design procedures that compute or select a spectral basis for model parameters, project post‑training parameter updates into that basis, and reparameterize weights so their effective update directions lie in a low‑dimensional, subspace‑aligned core. These methods remove or ignore orthogonal residual update directions, perform spectral parameter edits/rewiring, and produce compact parameter representations while analyzing the tradeoffs between compression and preserved post‑training performance.

subspace-alignedrewiring

Recent Skill Trend

Momentum and market value over time
Trending
Score
No comparison yet
-0.01
Oct 01, 2026Oct 01, 2026
Career
Value
No comparison yet
$200K/year
Oct 01, 2026Oct 01, 2026

Must-Read Papers

Most classic and influential ideas
View more

This study addresses the computational expense of traditional methods and the training inefficiency and instability of purely neural approaches for solving high-dimensional operator eigenvalue problems. To overcome these challenges, we propose the Stable Inverse Power Method Neural Network (SIPMNN). This method introduces a novel low-fidelity numerical spectral guidance mechanism, employing coarse-grid finite difference approximations to generate approximate eigenvalues as fixed shifts that guide and constrain the deep neural network training process. Experimental results on ten-dimensional benchmark problems demonstrate that SIPMNN achieves superior solution accuracy compared to purely neural baselines while reducing the required number of iterations by eight- to tenfold. Consequently, this work effectively resolves the bottlenecks associated with high-dimensional eigenvalue search difficulties and training instability.

finite difference methodhigh-dimensional problemsneural eigenvalue solver

In parametric dynamical systems, the Proper Orthogonal Decomposition (POD) basis drifts with parameters, degrading the accuracy of reduced-order models (ROMs). Method: This paper proposes the Projected Gaussian Process (pGP) framework—the first to formulate subspace adaptation as a statistical learning task mapping parameter space to the Grassmann manifold. It employs a two-stage geometric mapping: Euclidean space → horizontal space → Grassmann manifold, integrating POD, exponential/logarithmic maps, horizontal-space projection, and Gaussian process regression to enable uncertainty-aware POD subspace prediction while preserving manifold structure. Contribution/Results: Numerical experiments demonstrate that pGP significantly improves ROM accuracy and robustness in both parametric extrapolation and interpolation scenarios, and provides interpretable, calibrated confidence quantification—establishing a new paradigm for parameter-sensitive model reduction.

Adapting POD basis for parametric Reduced-Order ModelsMapping parameters to Grassmann manifold subspacesPredicting optimal subspaces using Gaussian Process regression

This work addresses the challenge of efficiently computing leading eigenvectors in dynamic graphs, where frequent updates to the adjacency or Laplacian matrix render traditional eigendecomposition methods computationally prohibitive. To overcome this limitation, the authors propose a fast spectral embedding update framework based on Rayleigh-Ritz projection. By leveraging eigenvector perturbation analysis, the method constructs a low-dimensional approximate invariant subspace that preserves high approximation accuracy while substantially reducing computational and memory costs. Experimental results demonstrate that the proposed approach outperforms existing techniques in both the quality of leading eigenvector approximation and performance on downstream tasks—such as influential node identification and node clustering—offering a compelling balance between efficiency and accuracy.

dynamic graphseigenvector updategraph evolution

This work addresses the challenge that real-world data in generalized linear models often violate the independent and identically distributed (i.i.d.) assumption. Under the relaxed assumption that the design matrix is orthogonally invariant—meaning its singular vectors are uniformly distributed while singular values remain arbitrary—the paper proposes an efficient parameter estimation method combining optimal spectral initialization with Approximate Message Passing (AMP). The proposed approach achieves the information-theoretically optimal sample complexity for weak recovery and attains the fundamental lower bound on estimation error, thereby extending beyond the classical i.i.d. Gaussian design setting. Rigorous theoretical analysis provides strong performance guarantees, and numerical experiments confirm both the algorithm’s effectiveness and the accuracy of the theoretical predictions on orthogonally invariant as well as more general correlated data.

generalized linear modelsorthogonally invariantparameter estimation

Spectral Estimators for Structured Generalized Linear Models via Approximate Message Passing

Aug 28, 2023
YZ
Yihan Zhang
🏛️ Institute of Science and Technology Austria | University of Cambridge

Parameter estimation in high-dimensional structured generalized linear models suffers from low efficiency, particularly under realistic design matrices exhibiting anisotropy and strong correlations. Method: This paper introduces a novel spectral estimation framework based on Approximate Message Passing (AMP). Contribution/Results: We provide the first exact asymptotic characterization of spectral estimators under correlated Gaussian designs. We identify a universally optimal covariance-adaptive preprocessing strategy, partially resolving a long-standing conjecture on optimal spectral estimation for rotationally invariant models. Theoretically and empirically, our approach substantially reduces sample complexity and achieves provably statistically optimal estimation accuracy—outperforming existing heuristic methods on canonical designs from computational imaging and genomics.

Characterizing spectral estimators for correlated Gaussian designsEstimating parameters in high-dimensional generalized linear modelsIdentifying optimal preprocessing for efficient parameter estimation

Latest Papers

What's happening recently
View more

This work addresses the computational bottleneck in traditional approaches for large-scale parametric dynamical systems, which require repeated and expensive eigenvalue solves to construct modal bases, thereby hindering efficient design optimization. To overcome this limitation, the authors propose a coupled architecture that integrates a rank-reduced autoencoder (RRAE) with a deep neural network. The RRAE leverages truncated singular value decomposition to define a unified low-dimensional parameter space, which serves as input to the neural network for jointly reconstructing all modal bases. This framework enables a nonlinear, physics-consistent parameterization of the modal basis while entirely eliminating the need for repeated eigenvalue computations. Validated on both 1D and 2D dynamical systems, the method demonstrates high accuracy and computational efficiency, effectively capturing dominant physical features and mitigating overfitting.

computational costdynamical systemseigenvalue problem

This work addresses the susceptibility of numerical Koopman spectral analysis to spurious eigenvalues, which compromises spectral reliability. To mitigate this issue, the authors propose a novel neural dictionary learning approach that uniquely formulates Koopman residual minimization as the central objective of dictionary learning and jointly optimizes the condition number of the data matrix to enhance numerical stability. The method simultaneously preserves one-step prediction accuracy while significantly improving the fidelity and certifiability of the Koopman spectrum. Experimental evaluations across diverse dynamical systems and real-world sea surface temperature data demonstrate that the proposed framework effectively suppresses spectral pollution, enhances pseudospectral inclusiveness, and yields superior overall approximation quality.

dictionary learningKoopman operatorresidual minimization

This study addresses the significant performance divergence among different prediction targets in flow matching, which arises from signal-to-noise ratio (SNR) discrepancies and information bottlenecks. By analyzing the SNR across individual directions of the data covariance, this work proposes a spectrum-mixing parameterization method that adapts to both time and direction. Departing from conventional manifold assumptions, it reveals that optimal parameterization depends on the directional SNR distribution rather than solely on intrinsic dimensionality. The theoretical optimality of this approach is rigorously proven on Gaussian data. Furthermore, the proposed method substantially accelerates optimization convergence without introducing additional training overhead while demonstrating strong robustness, thereby offering a novel theoretical perspective and practical framework for parameterization design in generative models.

Diffusion ModelsFlow MatchingInformation Bottleneck

This work addresses the limitations of conventional spectral neural operators, which rely on fixed global bases and struggle to capture spatial heterogeneity and multiscale dynamics. The authors propose the Adaptive Basis Learning (ABLE) framework—the first approach to enable end-to-end learning of spectral bases. ABLE constructs data-driven, spatially adaptive Parseval frames that preserve invertibility and maintain O(N log N) computational complexity, effectively shifting representational capacity from spectral coefficients to the basis functions themselves. The framework leverages an FFT-based efficient implementation, incorporates learnable auxiliary density functions, and can seamlessly replace spectral layers in existing neural operators. Experiments demonstrate that ABLE significantly outperforms strong baselines across multiple PDE benchmarks, particularly excelling in scenarios with sharp gradients and multiscale features. Moreover, when integrated as a plug-in module into models such as U-FNO and HPM, it consistently enhances performance.

adaptive basismultiscale dynamicsneural operators

This study addresses a fundamental challenge in system identification: distinguishing spurious eigenvalues arising from limited data from those genuinely reflecting the underlying system dynamics. To this end, the paper introduces—for the first time—the probabilistic sampling pseudospectrum \( P(\lambda) \) and its computationally efficient estimator \( \hat{P}(\lambda) \). By leveraging resampling and statistical inference, this framework quantifies the uncertainty of eigenvalues across the complex plane. The proposed approach provides a general and rigorous statistical criterion for data-driven methods such as Dynamic Mode Decomposition and subspace identification, substantially enhancing the reliability of identifying true dynamical modes from noisy, finite-length observations.

data-driven matriceseigenvalue artifactsfinite data error

Hot Scholars

LA

Lahari Anne

University of Illinois Urbana Champaign
MLAIData ScienceFull Stack Development
DT

Domenico Tortorella

Assistant professor (RTD-A), University of Pisa
machine learningdeep learning on graphsecho state networks
TW

Tandy Warnow

Grainger Distinguished Chair in Engineering, UIUC
Computer ScienceComputational BiologyPhylogeneticsMetagenomics
DS

Danial Saber

MSc, Ontario Tech University
Machine LearningGraph Neural Networks
JR

Jonathan Rubin

Research Scientist, Amazon Alexa AI
Deep LearningArtificial IntelligenceBiomedical Data AnalysisMachine Learning