gaussian integral calculus

This work investigates the joint uniform approximation capability of GELU-activated feedforward neural networks for multivariate polynomials, exponential functions, and reciprocal functions—along with their arbitrary-order derivatives. Addressing the lack of theoretical frameworks that simultaneously control high-order derivatives, we establish, for the first time, explicit uniform error bounds for both the target function and all its prescribed-order derivatives. We propose a constructive multiplicative approximation scheme and systematically extend it to division and exponentiation, ensuring global boundedness of all derivative orders. Through layerwise derivative control and asymptotic analysis at infinity, we derive explicit upper bounds on network width, weight magnitudes, and characterize the network’s behavior at infinity. Our results show that, for these fundamental function classes, the uniform approximation error in derivatives decays exponentially with network width.

This work investigates how feature learning affects neural scaling laws, specifically examining how task difficulty—categorized as hard, easy, or trivial—modulates performance scaling with model size, training steps, and dataset size. Methodologically, the study integrates infinite-width NTK theory, RKHS spectral analysis, empirical evaluation on nonlinear MLPs and CNNs, and power-law Fourier spectrum fitting. Theoretically, it establishes for the first time that feature learning approximately doubles the training-time and compute scaling exponents for hard tasks, and derives the computationally optimal scaling strategy. Experiments on function approximation on the circle and vision benchmarks confirm that feature learning markedly improves scaling efficiency for hard tasks, while yielding no gains for easy or trivial tasks. Theoretical predictions align closely with empirical results across all settings.

This work investigates the large-deviation behavior of deep neural networks with Gaussian i.i.d. weights under linearly growing (unbounded) activation functions—specifically ReLU. Addressing the limitation of existing large-deviation theory, which applies only to bounded continuous activations, we establish the first rigorous large-deviation principle for the ReLU case. Our method integrates tools from random matrix theory, Gaussian process analysis, and power series expansions to derive a concise, closed-form rate function. Crucially, we obtain an explicit power series representation of this rate function tailored to ReLU. The theoretical results align closely with empirical observations in modern deep learning architectures. This framework provides a novel analytical tool for quantitatively characterizing neural network generalization, informing principled weight initialization schemes, and elucidating training dynamics—thereby advancing the theoretical foundations of deep learning.

While theoretical equivalence between the Laplace kernel and the Neural Tangent Kernel (NTK) is established only in the infinite-width limit, empirical validation under realistic finite-width and high-dimensional Euclidean space (ℝᵈ) settings remains lacking. Method: This work conducts the first systematic regression experiments to assess their practical equivalence, employing two rigorous criteria: exact kernel function matching and consistency of Gaussian process posterior predictions—both evaluated under finite-width and ℝᵈ conditions. Contribution/Results: We demonstrate strong empirical agreement between the Laplace kernel and NTK in regression performance and generalization behavior, even when idealized assumptions (e.g., infinite width) are relaxed. This robust similarity provides solid empirical justification for approximating the computationally expensive NTK with the efficient Laplace kernel. The findings advance the practical deployment of lightweight kernel methods in deep learning modeling, bridging theoretical insights with scalable real-world applications.

Gaussian processes (GPs) struggle to rigorously incorporate uncountably infinite-dimensional functional prior information—such as boundary conditions or global physical constraints satisfied by PDE solutions. Method: This paper proposes a unified modeling framework grounded in reproducing kernel Hilbert spaces (RKHS), establishing for the first time a rigorous equivalence between the GP conditional expectation and orthogonal projection in RKHS. This enables direct embedding of functional constraints (e.g., Dirichlet or Neumann boundary conditions) into the GP prior, bypassing conventional pseudo-point approximations. Contribution/Results: We provide theoretical guarantees on existence, uniqueness, and convergence of the constrained GP posterior. Computationally, we design a practical numerical approximation algorithm. Experiments on PDE inverse problems demonstrate substantial improvements in uncertainty quantification accuracy and posterior consistency. The framework delivers a rigorous, general, and computationally tractable paradigm for integrating domain knowledge into Bayesian modeling.

This work addresses parametric numerical integration problems—including statistical functional evaluation, Chebyshev spectral approximation, and integrals arising from differential equations—by proposing a derivative-supervised differentiable machine learning framework. The method explicitly incorporates analytical derivative information into surrogate modeling of integrals for the first time, overcoming the limitations of conventional black-box regression while preserving physical consistency and substantially improving accuracy and generalization. Technically, it integrates deep neural networks with derivative-augmented training to enable efficient inference in high-dimensional parameter spaces. Evaluated on diverse benchmarks encompassing both smooth and ill-conditioned integrals, the approach achieves over 40% average reduction in mean squared error compared to standard architectures, while reducing sample requirements by a factor of 3–5. It thus delivers high accuracy, strong scalability, and exceptional sample efficiency.

This study investigates the impact of intrinsic data dimensionality on generalization in kernel methods, focusing on two notions of intrinsic dimension within kernel ridge regression: the Minkowski dimension $d_\rho$ induced by the kernel metric and the effective dimension $d_K$ derived from Kolmogorov $n$-widths. By establishing the first connection between $n$-widths and the eigenvalue decay of integral operators, the work reveals that $d_K$ can be substantially smaller than the geometric dimension $d_\rho$. The authors derive an excess risk bound of order $O(n^{-(2+d_K)/(2+2d_K)+\varepsilon})$ and propose a randomized sampling algorithm that, with high probability, estimates an upper bound on the $n$-widths using $O(\varepsilon^{-d_\rho} \log(1/\varepsilon))$ samples. Numerical experiments on fractal sets with Laplace kernels confirm that $d_K \ll d_\rho$, highlighting the practical advantage of the effective dimension.

Existing Gaussian process (GP) methods struggle to simultaneously achieve computational efficiency, predictive accuracy, and model customizability on large-scale datasets—primarily due to reliance on approximations that compromise uncertainty quantification, restrict kernel and noise model design, and hinder effective modeling of expressive nonstationary patterns. This paper introduces gp2Scale: a framework that automatically constructs sparse covariance structures via flexible, compactly supported nonstationary kernels, enabling exact GP inference on datasets with millions of observations without any approximation. By integrating distributed computing with advanced sparse linear algebra techniques, gp2Scale efficiently solves linear systems and computes log-determinants. The method supports arbitrary kernel functions, noise models, and input space types—including irregular or high-dimensional domains. Empirical evaluation across multiple real-world benchmarks demonstrates that gp2Scale significantly outperforms or matches state-of-the-art approximate GP methods, thereby overcoming a fundamental bottleneck in scalable exact GP modeling.

This work proposes a unified functional analytic framework that interprets both supervised and unsupervised learning as variational optimization problems within a function space induced by the data distribution. The central insight is that the fundamental distinction between these learning paradigms arises from the choice of the functional being optimized, rather than from differences in the underlying function space itself. Data structure is characterized via operators induced by the distribution, and target functions are estimated in the eigenbasis of these operators. This framework systematically integrates classical algorithms—including kernel methods, spectral clustering, and manifold learning—revealing their intrinsic coherence and underscoring the foundational role of function spaces and associated operators in modern machine learning.

This work addresses the challenge of jointly inferring multi-level functional states—such as curves, derivatives, and integrals—in functional data modeling, where existing methods struggle to account for derivative uncertainty, cross-level covariance, and identifiability of integration constants. The authors propose an anchored Gaussian process differential ensemble framework that explicitly models integration constants by embedding anchor points together with their mean-square derivatives and repeated integrals into a joint Gaussian state, enabling efficient computation via transformed Hilbert spaces. A key innovation is the separation of anchor-induced covariance from boundary uncertainty, revealing that integration constants cannot be uniquely identified from anchors alone. To enhance derivative recovery accuracy, the method introduces the TARTARE calibration strategy. Theoretical analysis combines Laplace–Dirichlet basis functions, finite-rank approximations, and operator-level approximation bounds. Experiments demonstrate substantially improved posterior derivative estimation in second-order simulations while preserving accuracy in anchors and integrals, and a motorcycle crash case study confirms coherent inference of coupled kinematic states and functional turning points.