neural operator learning

Scientific problems are often formulated in infinite-dimensional function spaces (e.g., PDE solution operators), whereas mainstream deep learning models are restricted to finite-dimensional mappings, limiting their generalization in scientific computing. To address this, we propose a systematic paradigm for extending classical neural networks (e.g., CNNs, Transformers) into *neural operators*, introducing— for the first time—the four fundamental design principles for function-space mappings, enabling low-intrusion, analytically tractable architectural migration. Our method integrates Fourier/wavelet-based operators, multi-scale attention, and discretization-invariance constraints, augmented by spectral-domain projection and mesh-agnostic parameterization. Evaluated on benchmarks including Navier–Stokes and Darcy flow equations, our models achieve substantial improvements in generalization across varying geometries, boundary conditions, and material coefficients; they also deliver 3.2× inference speedup and reduce generalization error by 47%.

Neural operators—deep models mapping between function spaces rather than vector spaces—lack open-source, discretization-agnostic implementations with theoretical convergence guarantees. To address this gap, we introduce NeuralOperator, the first modular and extensible Python library for neural operators built on PyTorch. It systematically supports state-of-the-art architectures—including Fourier Neural Operators (FNO) and Multipole Graph Neural Operators (MGNO)—and enables training and inference with functional inputs/outputs under diverse discretizations while rigorously ensuring discretization consistency and convergence. Through a unified API, comprehensive test coverage, and an end-to-end deployment toolchain, NeuralOperator significantly lowers the barrier to adopting neural operators in scientific computing tasks such as partial differential equation solving. The library bridges cutting-edge representational capacity with production-grade engineering robustness, making it both research-ready and deployable in real-world applications.

Existing attention mechanisms operate on discrete sequences, limiting their applicability to continuous function spaces essential for scientific machine learning tasks such as PDE solving and physical simulation. Method: This work generalizes attention to continuous function spaces by introducing the Transformer Neural Operator (TNO), the first rigorously defined attention mechanism on functions. It establishes a mathematically sound formulation of functional attention and proposes a patching-based continuous attention mechanism coupled with an efficient discretization strategy to mitigate computational complexity in high dimensions. Contribution/Results: TNO is proven to be a universal approximator for arbitrary continuous operators. Experiments demonstrate that it significantly outperforms state-of-the-art neural operators across diverse PDE benchmarks and physics-informed simulation tasks, validating its effectiveness, scalability, and generalization capability in scientific machine learning.

This work addresses aliasing errors introduced in the discrete implementation of Fourier Neural Operators (FNOs), which remain theoretically unquantified despite their practical significance. Specifically, the discrepancy between the continuous FNO formulation and its grid-based discretization has not been systematically characterized, and isolating this discretization error from other sources—such as approximation and optimization errors—remains challenging. Method: Leveraging tools from Fourier analysis, numerical functional analysis, and FFT theory, we derive an explicit algebraic convergence rate for the discretization error with respect to grid resolution and establish its quantitative dependence on the Sobolev regularity of the input function. Contribution/Results: We provide the first verifiable upper bound on this error, revealing intrinsic trade-offs among resolution, input smoothness, and model stability. Numerical experiments confirm the theoretical prediction: for inputs with higher Sobolev regularity, the error decays significantly under mesh refinement—thereby furnishing a rigorous foundation for reliable discrete FNO design.

Existing operator learning methods for irregular geometric domains suffer from high memory consumption and poor geometric adaptability. Method: This paper proposes the Deep Integral Operator (DIO) framework, which employs learnable compactly supported kernel functions and a sparsity-aware parametrization scheme to jointly ensure smoothness and computational efficiency; it further incorporates adaptive numerical quadrature to achieve geometry-agnostic modeling, eliminating reliance on structured grids. Contribution/Results: On standard benchmarks, DIO achieves higher accuracy than state-of-the-art neural operators, with improved training and test accuracy. It reduces trainable parameters by over an order of magnitude, significantly enhancing memory efficiency, generalization capability, and geometric robustness.

Modeling cross-dimensional physical quantities and their conservation laws on unstructured meshes with complex geometries remains highly challenging. This work proposes Topological Neural Operators (TNOs), which generalize neural operators to cell complexes for the first time by integrating discrete exterior calculus to explicitly model couplings among cells of varying dimensions through learnable gradient, curl, and divergence operators. TNOs decouple information propagation pathways from transformation mechanisms and incorporate hierarchically coarsened complexes to capture long-range dependencies and global topological structure. Experiments demonstrate that the method significantly outperforms existing approaches across multiple PDE benchmarks, including fluid dynamics problems on irregular geometries, thereby validating the efficacy of high-rank representations and native topological modeling.

This work addresses a key limitation of existing Transformer-based operator learning methods, which discretize continuous fields into independent tokens and thereby neglect the global structure of function spaces, hindering their ability to model mappings between infinite-dimensional functions. To overcome this, the authors propose Functional Attention—a novel mechanism inspired by geometric functional maps—that generalizes attention from pointwise affinities to linear functional correspondences over function spaces. By replacing the conventional softmax with a structured linear operator and integrating adaptive basis construction, the method explicitly captures global dependencies. The resulting representation is compact, generalizable, and resolution-invariant, achieving state-of-the-art performance on tasks such as partial differential equation solving, 3D segmentation, and regression, while demonstrating strong robustness across diverse discretization schemes.

This study addresses the limited accuracy of neural operators in solving partial differential equations characterized by sharp interfaces, heterogeneous coefficients, and multiscale structures. To this end, it proposes a localized operator learning framework based on a Partition of Unity (POU) Mixture-of-Experts. A geometry-aware gating network generates smooth spatial partitions to fuse local experts, while a novel HiRefPOU hierarchical residual architecture is introduced to achieve nested parent-child partitioning with global continuity. This design is further extended to Fourier Neural Operators to enhance spatial adaptivity. Evaluations on benchmarks such as Darcy flow demonstrate that the proposed method significantly outperforms global baselines. Moreover, the learned partitions exhibit strong interpretability, effectively improving both the accuracy and physical consistency of operator learning.

本文提出两种多阶段神经算子学习框架DCNO和DGNO，以解决卷积积分的快速准确计算问题，通过迭代优化算子近似提高精度。

该研究针对权重纠缠问题，提出权重操作符方法，通过两阶段学习过程实现功能组件的重用与独立适应。