ELM-DeepONets: Backpropagation-Free Training of Deep Operator Networks via Extreme Learning Machines

2025-01-16

AI Summary

To address the high computational cost and poor scalability of DeepONet training, this paper proposes ELM-DeepONet—the first operator learning framework integrating the Extreme Learning Machine (ELM) paradigm. It eliminates backpropagation entirely, reformulating training as a closed-form least-squares problem. Preserving the end-to-end architecture and theoretical soundness of DeepONet, ELM-DeepONet achieves gradient-free optimization with a single forward pass yielding globally optimal parameters. On benchmark nonlinear ODE and PDE operator modeling tasks, it matches or exceeds the accuracy of standard DeepONet while reducing training time by one to two orders of magnitude. This work substantially enhances the efficiency and scalability of operator learning for scientific computing, offering a novel, resource-efficient paradigm for complex physics-informed modeling—particularly valuable in compute-constrained settings.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Learning on the Edge & Model Compression
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Deep Operator Networks (DeepONets) are among the most prominent frameworks for operator learning, grounded in the universal approximation theorem for operators. However, training DeepONets typically requires significant computational resources. To address this limitation, we propose ELM-DeepONets, an Extreme Learning Machine (ELM) framework for DeepONets that leverages the backpropagation-free nature of ELM. By reformulating DeepONet training as a least-squares problem for newly introduced parameters, the ELM-DeepONet approach significantly reduces training complexity. Validation on benchmark problems, including nonlinear ODEs and PDEs, demonstrates that the proposed method not only achieves superior accuracy but also drastically reduces computational costs. This work offers a scalable and efficient alternative for operator learning in scientific computing.

Problem

• DeepONets
• computer resources
• complex pattern learning

Innovation

• ELM-DeepONets
• Reduced Training Time
• High Precision