sobolev space theory

This work addresses the absence of generalization error theory for multi-input neural operators in Sobolev spaces, particularly when input functions are defined on heterogeneous domains with differing dimensions and regularity. The paper establishes the first unified Sobolev generalization framework by integrating function approximation theory, Sobolev space analysis, and statistical learning theory. It derives complexity-dependent approximation and generalization error bounds of logarithmic-logarithmic over logarithmic type, quantifies the contribution of each input space to the overall error, and reveals the coupling mechanism among input dimensionality, regularity, and Sobolev smoothness order. The resulting theory is applicable to operator learning tasks in PDE solving and scientific computing, accurately characterizing the impact of multi-input interactions on learning performance under balanced settings.

This work investigates the generalization behavior of norm-minimizing interpolants in Sobolev spaces under noisy data, revealing a persistent and non-vanishing generalization error—referred to as benign overfitting—even in the large-sample regime. By introducing geometric arguments combined with Sobolev inequalities, the analysis is extended for the first time from Hilbert spaces (corresponding to \( p = 2 \)) to general Sobolev spaces with arbitrary \( p \in [1, \infty) \). The study identifies harmful neighborhoods near training points where interpolation amplifies noise. Under assumptions on label noise and data distribution regularity, it is shown that the generalization error of smoothness-preferring interpolants is, with high probability, bounded below by a positive constant. This underscores the critical role of function space selection in determining generalization performance.

This work addresses the lack of quantitative characterization of approximation capabilities of neural operators in Sobolev norms, which are crucial for the well-posedness, stability, and generalization of partial differential equations (PDEs). We establish, for the first time, a functional analytic framework for neural operators in Sobolev spaces, rigorously proving that their $H^t$ approximation error can be explicitly controlled by the number of network parameters and deriving a power-law relationship between the error and parameter complexity. Using Fourier Neural Operators (FNOs), an $H^1$-norm loss, and high-fidelity numerical simulations, we achieve an $H^1$ test error as low as $10^{-7}$ (corresponding to a relative error of approximately $10^{-3}$) on the Burgers equation. The experimental results align closely with theoretical predictions, validating the efficacy of the proposed framework.

This work investigates the convergence and learnability of stochastic gradient descent (SGD) for learning linear and nonlinear operators in general Hilbert spaces. To accommodate structural properties of target operators, we introduce weak and strong regularity conditions—functional analogues of classical smoothness and decay assumptions. Our analysis constitutes the first systematic extension of SGD convergence theory to infinite-dimensional operator learning. We establish that SGD converges to the optimal linear approximation of a nonlinear operator; derive tight non-asymptotic upper bounds on the convergence rate; and construct matching minimax lower bounds, thereby characterizing fundamental statistical limits. The results hold uniformly across both vector-valued and scalar-valued reproducing kernel Hilbert spaces (RKHS). By transcending the conventional restrictions of SGD analysis—namely, finite-dimensional parameterizations or linear models—this work provides the first theoretical framework for operator learning that incorporates functional regularity characterizations and delivers quantitative, provable solvability guarantees.

This work investigates the conditions and structural characteristics under which shallow neural networks achieve mathematical optimality in Banach function spaces. Method: For networks employing multivariate nonlinear activations—including ReLU, norm-based, and radial basis functions—we construct a novel Banach space grounded in k-plane transformations and sparse norms, and establish its reproducing kernel structure. Contribution/Results: We prove that the optimal approximation in this space is necessarily realized by a multivariate nonlinear neural architecture featuring skip connections, orthogonally normalized weights, and multi-index parameterization. This work establishes, for the first time, a rigorous correspondence between multivariate nonlinear neural networks and Banach-space optimality, unifying diverse classical activation functions within a single variational framework. It yields a universal representation theorem and delivers an analytic characterization of the optimal network structure. Moreover, it provides a rigorous theoretical foundation for regularity analysis and interpretable neural architecture design.

This work proposes a unified functional analytic framework that interprets both supervised and unsupervised learning as variational optimization problems within a function space induced by the data distribution. The central insight is that the fundamental distinction between these learning paradigms arises from the choice of the functional being optimized, rather than from differences in the underlying function space itself. Data structure is characterized via operators induced by the distribution, and target functions are estimated in the eigenbasis of these operators. This framework systematically integrates classical algorithms—including kernel methods, spectral clustering, and manifold learning—revealing their intrinsic coherence and underscoring the foundational role of function spaces and associated operators in modern machine learning.

This work addresses the statistical and computational challenges of learning bounded linear operators between Sobolev spaces from noisy data. The authors reformulate operator learning as an infinite-dimensional matrix regression in wavelet coordinates and propose a finite-resolution, block-wise least squares estimator that exploits multiscale structure. To account for the heterogeneous local estimation difficulty across scales, they introduce a scale-adaptive sampling strategy. Under the Sobolev operator norm loss, the proposed method achieves the minimax optimal convergence rate while attaining the best possible computational complexity among dense least squares–type algorithms.

This study investigates the theoretical foundations of operator learning, with a focus on convergence rates and fundamental statistical limits. By integrating tools from statistical learning theory, approximation theory, and the framework of holomorphic operators, the work establishes a unified error analysis framework to systematically derive generalization error bounds for empirical risk minimization. Under generalized regularity conditions, it further establishes minimax-optimal statistical lower bounds, revealing an intrinsic trade-off between sample complexity and model approximation capacity. The analysis delineates current theoretical boundaries in operator learning and identifies several key open problems, offering new perspectives to guide future theoretical advances in the field.

Existing neural operators lack reliability and theoretical guarantees when handling out-of-distribution input functions. This work proposes an extended framework grounded in reproducing kernel Hilbert spaces (RKHS), leveraging kernel approximation techniques to achieve robust approximation of both out-of-distribution functions and their derivatives. The key innovation lies in establishing a theoretical connection between kernel selection and Sobolev eigenfunction spaces, thereby providing predictable guarantees on generalization error and derivative accuracy for neural operators. When applied to solving elliptic partial differential equations—particularly on manifolds represented as point clouds—the method demonstrates significantly enhanced geometric awareness, improved extrapolation accuracy, and greater computational efficiency.

This work addresses the limitation of classical reproducing kernel Hilbert spaces (RKHS) in modeling learning architectures with non-Hilbertian geometric structures—such as fixed-architecture neural networks equipped with non-quadratic norms—by developing a functional-analytic framework for reproducing kernel Banach spaces (RKBS) with feature maps. Through the introduction of structural conditions, the authors recover key components including feature mappings, kernel construction, and a representer theorem, thereby formulating supervised learning as either a minimum-norm interpolation or a regularized optimization problem. This study establishes, for the first time, a theoretically sound RKBS framework in non-Hilbertian Banach spaces that supports both feature representation and kernel-based learning. It demonstrates that fixed-architecture neural networks naturally induce such spaces, unifying kernel methods and neural networks under a common function-space perspective and significantly extending the applicability of kernel learning principles.