functional analysis

This work investigates the universal approximation capabilities of Transformers and neural integral operators in Banach spaces. Addressing three core problems: (1) whether Transformers can universally approximate integral operators between Hölder spaces; (2) whether neural integral operators exist that universally approximate arbitrary continuous linear or nonlinear operators between Banach spaces; and (3) how to overcome regularity constraints for broader approximation. We first establish, for the first time, the universal approximation property of Transformers for integral operators acting between Hölder spaces. Second, we propose a generalized neural integral operator based on the Gavurin integral and prove its universal approximation theorem for continuous operators between arbitrary Banach spaces. Third, we incorporate Leray–Schauder mappings into the Transformer architecture to eliminate dependence on smoothness assumptions on input/output spaces. These results provide a rigorous functional-analytic foundation for operator learning and extend the theoretical scope of deep learning in infinite-dimensional tasks such as PDE solving and physics-informed modeling.

This paper addresses the universal approximation of continuous (including nonlinear) operators on Banach spaces. Methodologically, it introduces a novel learning framework based on orthogonal polynomial projections—marking the first integration of Leray–Schauder mapping theory into operator approximation theorems, synergizing Banach-space operator analysis with spectral approximation techniques in $L^p$ spaces. Specifically, in $L^p$ (notably $L^2$), it establishes a two-stage operator learning paradigm: “learnable projection” followed by “finite-dimensional mapping.” Theoretical contributions include: (1) a proof of universal approximation capability for the framework on arbitrary Banach spaces; (2) explicit sufficient conditions ensuring high-precision operator approximation in $L^2$; and (3) the first rigorous, unified mathematical foundation for operator neural networks.

This paper investigates necessary and sufficient conditions for the composition operator (C_h f = f circ h) to map Riesz bases or orthonormal bases between (L^2(Omega_1)) and (L^2(Omega_2)). Focusing on the geometric perturbation induced by the mapping (h: Omega_2 o Omega_1) on standard orthonormal bases, we establish the first complete characterization: (C_h) preserves Riesz bases—i.e., maps any Riesz basis of (L^2(Omega_1)) onto a Riesz basis of (L^2(Omega_2))—if and only if (h) is a bi-Lipschitz bijection. Moreover, (C_h) preserves orthonormal bases if and only if (h) is additionally measure-preserving (up to equivalence). This equivalence reveals a fundamental connection between geometric regularity of (h) and stability of basis structures under composition. Building upon this characterization, we propose a novel paradigm for constructing complete, optimally approximating sequences via bijective neural networks. Our framework provides a rigorous functional-analytic foundation for invertible neural network approximation and yields verifiable constructive criteria for basis preservation.

Global approximation of temporal functions—such as path functionals—in large-scale or infinite-dimensional weighted spaces remains challenging, particularly due to the limitations of conventional compact-support assumptions. Method: We propose functional neural networks tailored for weighted spaces, featuring additive mappings, scalar-valued activations, and linear readouts. This architecture circumvents traditional compact-support constraints and unifies the modeling of signature-based linear functionals and non-anticipative path functionals. We further establish an equivalence between the reproducing kernel Hilbert space (RKHS) induced by the signature kernel and the Cameron–Martin space of associated Gaussian processes. Contributions: First, we prove a universal approximation theorem: additive networks globally approximate continuous functions on weighted spaces. Second, we demonstrate that signature-based linear functionals are universally approximable within this framework. Third, we provide a Gaussian-process-based theoretical foundation for uncertainty quantification in signature kernel regression.

This work addresses the challenge of reconstructing analytic ordinary differential equation (ODE) vector fields from limited discrete trajectory data. Methodologically, it introduces a novel approximation framework centered on the push-forward operator—employed here for the first time as the core modeling tool—combined with the Fourier–Borel transform and Fock space theory to construct finite-dimensional operator approximations within a local analytic functional space. Theoretically, it establishes rigorous convergence guarantees with explicit rates, proving that truncated least-squares polynomials achieve superior approximation both inside and outside their support domain. Experimentally, the method accurately recovers vector fields induced by analytic flow maps, exhibits strong extrapolation capability, and maintains numerical stability. Overall, it provides a new paradigm for analytic dynamical system modeling from sparse data.

This work addresses the problem of efficient, dimension-independent approximation by encoder–decoder neural operators through a novel theoretical framework grounded in variational spaces. The approach characterizes classes of nonlinear operators via vector-valued measures on input and output function spaces, thereby circumventing restrictive assumptions such as Lipschitz continuity or Fréchet differentiability commonly imposed in prior analyses. Leveraging Bochner space techniques, the study establishes an $L^q$-norm approximation error bound for two-layer encoder–decoder networks, where the bounding constant is independent of the encoding dimension. The error decomposes into contributions from input encoding, output encoding, and a finite-width term scaling as $N^{-1/2}$. When the encoding errors decay polynomially, the overall scheme achieves algebraic approximation rates and corresponding learning rates.

This work addresses the limitation of classical reproducing kernel Hilbert spaces (RKHS) in modeling learning architectures with non-Hilbertian geometric structures—such as fixed-architecture neural networks equipped with non-quadratic norms—by developing a functional-analytic framework for reproducing kernel Banach spaces (RKBS) with feature maps. Through the introduction of structural conditions, the authors recover key components including feature mappings, kernel construction, and a representer theorem, thereby formulating supervised learning as either a minimum-norm interpolation or a regularized optimization problem. This study establishes, for the first time, a theoretically sound RKBS framework in non-Hilbertian Banach spaces that supports both feature representation and kernel-based learning. It demonstrates that fixed-architecture neural networks naturally induce such spaces, unifying kernel methods and neural networks under a common function-space perspective and significantly extending the applicability of kernel learning principles.

This work addresses the challenge of stably and efficiently approximating nonlinear operators between function spaces under varying discretizations and output domains using Transformer architectures. To this end, it proposes a graph-preserving Functional Graph Transformer that lifts input functions into graph-measure representations, formulating operator learning within a measure-theoretic framework. The graph-preserving structure guarantees that outputs remain single-valued functions while naturally supporting discretization refinement and cross-domain queries. This approach is the first to unify the treatment of negative-order Sobolev inputs, positional encoding effects, and discretization consistency. Theoretically, it establishes that a finite-depth self-attention mechanism combined with MLPs can universally approximate a broad class of nonlinear operators, achieving both expressive power and generalization consistency across discretizations.

This work addresses a fundamental limitation of conventional operator learning methods, which rely on uniform or $L^p$ approximation frameworks and struggle to handle discontinuous or set-valued operators—such as maximally monotone differential operators. To overcome this challenge, the paper introduces, for the first time, a novel approximation paradigm grounded in graph convergence (specifically, Painlevé–Kuratowski convergence). The proposed approach employs an encoder–decoder neural network architecture combined with resolvent-based parameterization to achieve continuous approximation in the sense of local graph convergence, while rigorously preserving maximal monotonicity. By doing so, this study transcends the constraints of classical approximation theory and establishes a structure-preserving theoretical framework for learning operators with inherent set-valued or discontinuous characteristics.

This work addresses the universal approximation problem for nonlinear $k$-times differentiable operators and their derivatives in infinite-dimensional Banach spaces. By leveraging an encoder–decoder architecture—encompassing models such as DeepONets—and integrating Bastiani differentiability, the compact-open topology, and a novel weighted Sobolev space framework, we extend classical universal approximation theorems to the setting of infinite-dimensional operator learning for the first time. We establish the first universal approximation theorem guaranteeing uniform approximation of nonlinear operators and all their derivatives up to order $k$ on compact sets, under a broad class of finite input measures. This result provides a rigorous theoretical foundation for high-order-accuracy operator learning, numerical solution of infinite-dimensional PDEs, and optimization problems constrained in Banach spaces.