Technology Category
Machine Learning: Neuro-Symbolic LearningSearch and Optimization: Learning to SearchComputer Vision: Learning & Optimization for CV
Self-Composing Neural Operators with Depth and Accuracy Scaling via Adaptive Train-and-Unroll Approach
🤖 AI Summary
In large-scale scientific machine learning, neural operators often struggle to simultaneously achieve high efficiency and accuracy. Method: This paper proposes the Self-Composing Neural Operator (SCNO) framework, which employs iterative deep unfolding of a single operator block, integrating a multigrid-inspired backbone network with an adaptive training strategy featuring progressive depth growth. SCNO explicitly leverages a scaling law governing accuracy versus depth to dynamically co-optimize model capacity and computational resources. Contributions/Results: SCNO achieves state-of-the-art performance on standard PDE-solving benchmarks. Moreover, in high-frequency ultrasound computed tomography, it significantly improves modeling accuracy and generalization for wave propagation phenomena—demonstrating its effectiveness and scalability for high-oscillation physical modeling.
Technology Category
Application Category
📝 Abstract
In this work, we propose a novel framework to enhance the efficiency and accuracy of neural operators through self-composition, offering both theoretical guarantees and practical benefits. Inspired by iterative methods in solving numerical partial differential equations (PDEs), we design a specific neural operator by repeatedly applying a single neural operator block, we progressively deepen the model without explicitly adding new blocks, improving the model's capacity. To train these models efficiently, we introduce an adaptive train-and-unroll approach, where the depth of the neural operator is gradually increased during training. This approach reveals an accuracy scaling law with model depth and offers significant computational savings through our adaptive training strategy. Our architecture achieves state-of-the-art (SOTA) performance on standard benchmarks. We further demonstrate its efficacy on a challenging high-frequency ultrasound computed tomography (USCT) problem, where a multigrid-inspired backbone enables superior performance in resolving complex wave phenomena. The proposed framework provides a computationally tractable, accurate, and scalable solution for large-scale data-driven scientific machine learning applications.
Problem
Research questions and friction points this paper is trying to address.
Enhancing neural operator efficiency and accuracy via self-composition
Training progressively deepening models with adaptive train-and-unroll approach
Solving large-scale data-driven scientific machine learning applications
Innovation
Methods, ideas, or system contributions that make the work stand out.
Self-composing neural operators via iterative block application
Adaptive train-and-unroll approach for depth scaling
Multigrid-inspired backbone for complex wave resolution
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
No related jobs found.
