HyperNOs: Automated and Parallel Library for Neural Operators Research

2025-03-23

AI Summary

To address the lack of automated, systematic tools for hyperparameter optimization in neural operators, this paper proposes the first fully automated, parallelized hyperparameter search framework specifically designed for neural operators. Built upon PyTorch and integrated with Ray-Tune for distributed optimization, the framework supports joint hyperparameter tuning across diverse operator architectures (e.g., Fourier-based and CNN-based), datasets, and spatial/temporal resolutions, while enforcing a fixed parameter-count constraint during training. It provides extensible, modular interfaces that balance usability and customization, enabling user-defined models and datasets. Evaluated on multiple PDE benchmark tasks—including Burgers’, Darcy, and Navier–Stokes equations—the framework achieves state-of-the-art performance, demonstrating its efficacy in modeling real-world physical data and modern neural operator architectures. By significantly reducing implementation complexity and experimental overhead, it lowers the barrier to entry for neural operator research and deployment.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Auto ML and Hyperparameter Tuning
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

This paper introduces HyperNOs, a PyTorch library designed to streamline and automate the process of exploring neural operators, with a special focus on hyperparameter optimization for comprehensive and exhaustive exploration. Indeed, HyperNOs takes advantage of state-of-the-art optimization algorithms and parallel computing implemented in the Ray-tune library to efficiently explore the hyperparameter space of neural operators. We also implement many useful functionalities for studying neural operators with a user-friendly interface, such as the possibility to train the model with a fixed number of parameters or to train the model with multiple datasets and different resolutions. We integrate Fourier neural operators and convolutional neural operators in our library, achieving state of the art results on many representative benchmarks, demonstrating the capabilities of HyperNOs to handle real datasets and modern architectures. The library is designed to be easy to use with the provided model and datasets, but also to be easily extended to use new datasets and custom neural operator architectures.

Problem

• Automates hyperparameter optimization for neural operators
• Enables parallel computing for efficient hyperparameter space exploration
• Provides user-friendly interface for training and extending neural operators

Innovation

• Automated hyperparameter optimization with Ray-tune
• Parallel computing for efficient space exploration
• User-friendly interface for diverse neural operators