subspace projection

Traditional neural networks suffer from limited interpretability and weak theoretical foundations. Method: This paper proposes a novel machine learning paradigm grounded in infinite-dimensional Hilbert spaces, centering on linear operators. It integrates reproducing kernel Hilbert spaces (RKHS), spectral operator learning, wavelet representations, scattering transforms, and Koopman operator theory to formulate learning tasks as sampling, approximation, and dynamical inference in infinite-dimensional function spaces. Contribution/Results: We establish the first unified Hilbert-space-theoretic framework bridging spectral learning and symbolic reasoning. The approach significantly enhances mathematical rigor and model interpretability by grounding learning in well-defined functional-analytic principles. Moreover, it provides a rigorous mathematical foundation and new methodological pathways for deep interdisciplinary integration between signal processing and machine learning—enabling principled analysis of structured data, hierarchical feature extraction, and nonlinear dynamical system modeling.

This work addresses the learning of nonlinear operators between Banach spaces. Methodologically, it introduces the first general approximation framework for operator learning grounded in Leray–Schauder mapping theory—incorporating compact operator theory and finite-dimensional projections—to construct provably convergent approximations within compact subspaces, without assuming any specific parametric form. The framework guarantees uniform approximation for a broad class of nonlinear operators, underpinned by rigorous functional-analytic guarantees. Evaluated on two standard PDE-solving benchmarks, the proposed approach achieves accuracy comparable to state-of-the-art models, confirming both its theoretical soundness and empirical effectiveness. The core contribution is the establishment of the first fixed-point-theoretic paradigm for operator learning, endowed with functional-analytic verifiability: it provides principled, non-empirical justification for approximation capability, bridging deep operator learning with classical nonlinear functional analysis.

This paper addresses the universal approximation of continuous (including nonlinear) operators on Banach spaces. Methodologically, it introduces a novel learning framework based on orthogonal polynomial projections—marking the first integration of Leray–Schauder mapping theory into operator approximation theorems, synergizing Banach-space operator analysis with spectral approximation techniques in $L^p$ spaces. Specifically, in $L^p$ (notably $L^2$), it establishes a two-stage operator learning paradigm: “learnable projection” followed by “finite-dimensional mapping.” Theoretical contributions include: (1) a proof of universal approximation capability for the framework on arbitrary Banach spaces; (2) explicit sufficient conditions ensuring high-precision operator approximation in $L^2$; and (3) the first rigorous, unified mathematical foundation for operator neural networks.

Existing neural operators lack reliability and theoretical guarantees when handling out-of-distribution input functions. This work proposes an extended framework grounded in reproducing kernel Hilbert spaces (RKHS), leveraging kernel approximation techniques to achieve robust approximation of both out-of-distribution functions and their derivatives. The key innovation lies in establishing a theoretical connection between kernel selection and Sobolev eigenfunction spaces, thereby providing predictable guarantees on generalization error and derivative accuracy for neural operators. When applied to solving elliptic partial differential equations—particularly on manifolds represented as point clouds—the method demonstrates significantly enhanced geometric awareness, improved extrapolation accuracy, and greater computational efficiency.

To address weak class discriminability and insufficient feature robustness in machine learning, this paper proposes a Multi-level Orthogonal Subspace (MOS) Karhunen–Loève feature theory within a random tensor space. Training data are modeled as stochastic processes in a Bochner space, and hierarchical KL expansions explicitly decouple dominant class structures from inter-class anomalous signals, enabling class-wise subspace disentanglement and interpretable projection features. This work establishes, for the first time, a MOS feature construction paradigm under the random tensor framework—uniquely integrating statistical modeling rigor with geometric interpretability. Evaluated on the ADNI plasma dataset, the method significantly outperforms gradient boosting, RUS Boost, random forests, and CNNs, achieving substantial gains in classification accuracy. These results validate its robust discriminative capability for high-noise biomedical data.

Existing data-driven Koopman operator methods struggle to ensure approximate invariance of subspaces under the operator in non-Euclidean settings, limiting predictive accuracy. This work addresses this challenge by extending principal vector–guided subspace pruning to reproducing kernel Hilbert spaces (RKHS) for the first time. By precisely computing principal angles and vectors in RKHS, we introduce Kernel-SPV and its computationally efficient Nyström approximation–based variant, Approximate Kernel-SPV. These approaches overcome the limitations of traditional Euclidean formulations, significantly enhancing the invariance of Koopman-invariant subspaces while maintaining scalability and substantially improving prediction accuracy.

This work addresses the limitations of conventional spectral neural operators, which rely on fixed global bases and struggle to capture spatial heterogeneity and multiscale dynamics. The authors propose the Adaptive Basis Learning (ABLE) framework—the first approach to enable end-to-end learning of spectral bases. ABLE constructs data-driven, spatially adaptive Parseval frames that preserve invertibility and maintain O(N log N) computational complexity, effectively shifting representational capacity from spectral coefficients to the basis functions themselves. The framework leverages an FFT-based efficient implementation, incorporates learnable auxiliary density functions, and can seamlessly replace spectral layers in existing neural operators. Experiments demonstrate that ABLE significantly outperforms strong baselines across multiple PDE benchmarks, particularly excelling in scenarios with sharp gradients and multiscale features. Moreover, when integrated as a plug-in module into models such as U-FNO and HPM, it consistently enhances performance.

This work addresses the heavy reliance on large numbers of PDE solution samples in operator learning by proposing an efficient approximation framework based on incremental dimensionality expansion. By integrating tensor-product basis expansions with sparse recovery techniques—such as orthogonal matching pursuit—the method identifies low-dimensional structures and critical variable interactions, substantially reducing the required number of PDE solves. In multiple numerical experiments, the approach achieves accuracy comparable to or better than conventional quadrature methods and Fourier neural operators, while demanding significantly lower sample complexity and computational cost. Furthermore, the recovered sparse index sets offer interpretable insights into the underlying solution structure.

This work addresses the longstanding challenge in operator learning of simultaneously achieving stability, interpretability, and efficient parallelization, which stems from the lack of explicit spectral structure modeling. The authors propose a polar-spectral operator framework that leverages polar-coordinate geometry to map problems into the spectral domain, where they are decomposed into orthogonal eigenmodes processed independently. A self-adjoint-inspired spectral constraint mechanism is introduced, which not only reduces parameter count and computational complexity but also naturally yields a novel mode-based model parallelization strategy. Experiments demonstrate that the method enables stable training on MNIST, significantly improves convergence, and produces more interpretable and computationally efficient model representations.

This work proposes Neural-POD, a novel framework that overcomes the limitations of traditional AI-for-Science approaches, which often fail to generalize across new parameters or discretizations due to dependence on fixed grids or resolutions. By constructing nonlinear orthogonal bases in infinite-dimensional function spaces via neural networks, Neural-POD reformulates basis construction as a sequence of residual minimization problems, analogous to a nonlinear, learnable Gram–Schmidt process that incrementally captures data structure. The method transcends the linearity constraints of classical Proper Orthogonal Decomposition (POD), enabling optimization under arbitrary norms, resolution-invariant mappings, and effective nonlinear feature extraction. It is designed for seamless integration into reduced-order modeling and operator learning pipelines. Numerical experiments on complex spatiotemporal systems—including the Burgers and Navier–Stokes equations—demonstrate its robustness and efficacy in bridging classical model reduction with modern operator learning paradigms.