spectral-entropy regularization

Traditional neural networks suffer from limited interpretability and weak theoretical foundations. Method: This paper proposes a novel machine learning paradigm grounded in infinite-dimensional Hilbert spaces, centering on linear operators. It integrates reproducing kernel Hilbert spaces (RKHS), spectral operator learning, wavelet representations, scattering transforms, and Koopman operator theory to formulate learning tasks as sampling, approximation, and dynamical inference in infinite-dimensional function spaces. Contribution/Results: We establish the first unified Hilbert-space-theoretic framework bridging spectral learning and symbolic reasoning. The approach significantly enhances mathematical rigor and model interpretability by grounding learning in well-defined functional-analytic principles. Moreover, it provides a rigorous mathematical foundation and new methodological pathways for deep interdisciplinary integration between signal processing and machine learning—enabling principled analysis of structured data, hierarchical feature extraction, and nonlinear dynamical system modeling.

This paper addresses the detection and estimation of rank-one signals with directional priors (e.g., nonnegative signal components) from noisy observations. Conventional PCA and spectral methods suffer from fundamental limitations in this setting. To overcome them, we propose a nonlinear Laplacian matrix construction: given the observation matrix $ Y $, we apply a nonlinear activation function $ sigma $ to the degree vector $ Ymathbf{1} $, incorporate the resulting diagonal correction term, and extract the leading eigenvector. We establish, for the first time, a rigorous theoretical framework coupling nonlinear Laplacian spectral analysis with directional priors, precisely characterizing how the detectability threshold—the critical signal-to-noise ratio—depends on $ sigma $. The method is non-iterative, tunable, computationally efficient, and robust. Theory shows a significant reduction in the critical SNR; on models such as Gaussian planted submatrix, it outperforms standard spectral algorithms and even sophisticated iterative methods like AMPA under optimal $ sigma $, achieving both statistical optimality and computational simplicity.

Hyperspectral imaging (HSI) suffers from high spectral dimensionality and substantial redundancy, necessitating dimensionality reduction methods that jointly optimize predictive performance and interpretability. This paper proposes a post-hoc interpretable band selection framework for HSI: it introduces the deletion-insertion evaluation paradigm to HSI band selection for the first time, quantifying each band’s influence score on a pre-trained classifier’s decision. High-impact bands with clear physical meaning are then selected. The method integrates model-aligned contribution analysis, influence score aggregation, and empirical validation. Experiments on the Pavia University and Salinas datasets demonstrate that classifiers using only 30 selected bands achieve accuracy comparable to or exceeding that of full-spectrum models, while significantly reducing computational cost. Crucially, the approach ensures both interpretability—through transparent, physics-grounded band selection—and generalizability across diverse scenes and classifiers.

Parameter estimation in high-dimensional structured generalized linear models suffers from low efficiency, particularly under realistic design matrices exhibiting anisotropy and strong correlations. Method: This paper introduces a novel spectral estimation framework based on Approximate Message Passing (AMP). Contribution/Results: We provide the first exact asymptotic characterization of spectral estimators under correlated Gaussian designs. We identify a universally optimal covariance-adaptive preprocessing strategy, partially resolving a long-standing conjecture on optimal spectral estimation for rotationally invariant models. Theoretically and empirically, our approach substantially reduces sample complexity and achieves provably statistically optimal estimation accuracy—outperforming existing heuristic methods on canonical designs from computational imaging and genomics.

This work addresses the bias in heavy-tailedness estimation arising from aspect-ratio disparities in weight matrices during spectral analysis of deep neural networks. We propose Fixed-Aspect-Ratio Matrix Sampling (FARMS), a method that mitigates this bias by randomly sampling submatrices with fixed aspect ratios, modeling their empirical spectral density (ESD), and fitting α-stable distributions to estimate the tail index. FARMS is the first framework to systematically eliminate the intrinsic aspect-ratio-induced bias in spectral statistics. It exhibits strong cross-architecture and cross-task robustness, significantly improving model diagnostics and layer-wise hyperparameter allocation. Extensive validation across computer vision (CV), scientific machine learning (SciML), and large language model (LLM) pruning tasks confirms its effectiveness: when applied to LLaMA-7B pruning, FARMS reduces perplexity by 17.3%, outperforming state-of-the-art methods.

This work addresses the limitations of conventional neural networks in structural transparency and controllability of functional shape by introducing a spectral neuron model. The proposed approach leverages input-driven real symmetric affine matrices, using their eigenvalues as outputs to integrate spectral theory with matrix-based learning. While preserving strong nonlinear representational capacity, the model explicitly embeds mathematical structure, enabling explicit shape constraints such as convexity or concavity. Robust training is achieved through semidefinite optimization. Theoretical analysis and empirical experiments demonstrate that the model simultaneously offers interpretability, learnability, scalability, and precise control over the geometric properties of learned functions.

This work addresses the longstanding challenge in operator learning of simultaneously achieving stability, interpretability, and efficient parallelization, which stems from the lack of explicit spectral structure modeling. The authors propose a polar-spectral operator framework that leverages polar-coordinate geometry to map problems into the spectral domain, where they are decomposed into orthogonal eigenmodes processed independently. A self-adjoint-inspired spectral constraint mechanism is introduced, which not only reduces parameter count and computational complexity but also naturally yields a novel mode-based model parallelization strategy. Experiments demonstrate that the method enables stable training on MNIST, significantly improves convergence, and produces more interpretable and computationally efficient model representations.

This work addresses the gap between algorithmic prototypes and efficient implementations in scientific research by proposing a lightweight approach to translate statistical and machine learning algorithms—such as kernel ridge regression and stochastic gradient descent matrix factorization—from mathematical formulations into readable, high-performance C++ code. Leveraging the Eigen template library for core linear algebra operations—including kernel matrix construction, regularized solvers, and vectorized updates—the implementation seamlessly integrates into the Python ecosystem via pybind11, enabling efficient interoperability with NumPy arrays. The project provides concise, reproducible code examples that encapsulate common computational patterns in research, significantly lowering the barrier for researchers to adopt C++ for high-performance development while balancing performance, readability, and usability.

This work investigates the identification of shared and view-specific latent structures in multi-view data, focusing on scenarios where symmetric positive definite matrices exhibit common eigendirections. By analyzing the spectral properties of matrix interpolants of the form \(A^{1-x}B^x\), the study establishes that the operator norm displays strict or approximate log-linear behavior if and only if the matrices share dominant eigenvectors. Leveraging this theoretical insight, the authors develop a multi-manifold learning framework that integrates complex matrix interpolation, spectral analysis, and singular vector alignment to effectively disentangle common and specific latent variables across views. This approach provides a novel theoretical foundation and a practical methodology for multi-view representation learning.

This work addresses the instability of reconstruction in ill-posed inverse problems caused by noise, as well as limitations of conventional regularization methods—such as reliance on manual hyperparameter tuning—and the lack of interpretability and cross-resolution generalization in deep learning approaches. To this end, we propose the Spectral Correction Network (SC-Net), which learns a signal-to-noise-ratio-adaptive, pointwise filtering function in the spectral domain of the forward operator to reweight spectral coefficients, yielding stable and interpretable solutions. Our method uniquely integrates interpretable adaptive spectral filtering with operator learning, and we theoretically prove that it can approximate continuous inverse operators, possesses discretization invariance, and achieves minimax optimal convergence rates. Experiments on 1D integral equations demonstrate a convergence rate of $O(\delta^{0.5})$, matching theoretical optimality; the learned filter outperforms Oracle Tikhonov regularization and generalizes zero-shot from $N=256$ to $N=2048$ with reconstruction error maintained at approximately 0.23.