SINO: Scale-Invariant Neural Operator

2026-09-29

AI Summary

This study addresses the challenges of mesh-resolution dependence and closure in scientific machine learning by proposing scale-invariant neural operators. The method employs a dual-branch spectral-spatial architecture that learns over normalized physical scales to prevent the memorization of grid-specific patterns. Furthermore, it introduces explicit low-rank parameterization and bottleneck MLPs to generate continuous convolution kernels as an inductive bias, achieving over 95% variance compression while accurately capturing scale-invariant physical laws. Experimental results demonstrate that, compared to baseline models, this approach reduces errors by 1.5 to 38 times, improves parameter efficiency by 2 to 23 times, and steepens the scaling law exponent by a factor of 38, thereby significantly enhancing both generalization capability and computational efficacy.

Abstract

In scientific machine learning, physical fields governed by partial differential equations exhibit low-rank structure and scale invariance. When solving equations on coarse grids, missing information leads to the closure problem: modeling unresolved physics to recover lost dynamics. Although closure terms depend on grid resolution, they represent scale-invariant physical laws. A model truly learning physics should capture these mechanisms with low-rank parameterization rather than memorizing grid-specific patterns. Inspired by this, we propose the Scale-Invariant Neural Operator (SINO), which learns on normalized physical scales via a dual-branch architecture operating in spectral and spatial domains. SINO uses bottleneck MLPs to generate continuous convolution kernels, embedding an explicit low-rank inductive bias that concentrates more than 95 percent of variance in 2-3 modes, as validated by PCA across benchmarks, while drastically reducing parameters. This principled design yields 38 times steeper scaling law exponents than FNO, demonstrating superior parameter efficiency. We compare SINO with traditional models (U-Net, DeepONet), Transformer models (Transolver, Oformer, GK-Transformer), and frequency-domain models (FNO, AMFNO, UFNO) on closure problems spanning externally forced Burgers turbulence, decaying Burgers turbulence, KS turbulence, Kolmogorov-forced NS turbulence, and decaying NS turbulence. Experiments show SINO achieves 1.5-38 times error reduction and 2-23 times parameter efficiency over baselines, with superior scaling laws reflecting exceptional data efficiency from principled low-rank design. Code is available at https://github.com/AI4Science-WestlakeU/SINO.

Problem

• closure problem
• scale invariance
• partial differential equations
• low-rank structure
• scientific machine learning

Innovation

• Scale-Invariant Neural Operator
• Closure Problem
• Low-Rank Inductive Bias
• Dual-Branch Architecture
• Scaling Law