Low-rank adaptive physics-informed HyperDeepONets for solving differential equations

2025-07-24

AI Summary

To address the high memory and computational overhead of HyperDeepONets caused by their large output parameter count, this paper proposes Physics-Informed Low-Rank Adaptive HyperDeepONets (PI-LRA-HyperDeepONet). Methodologically, we decompose the hypernetwork’s output layer into low-rank factors and incorporate an adapter-style low-rank update mechanism; concurrently, physics-informed regularization is embedded into the backbone network to jointly enhance expressivity and generalization. Our key contribution is the first integration of low-rank adaptation with physics-informed neural networks within the operator learning framework. Experiments on ordinary and partial differential equation solving tasks demonstrate that PI-LRA-HyperDeepONet reduces model parameters by 70% while significantly outperforming baseline models in prediction accuracy and generalization. Moreover, it substantially compresses model complexity and inference cost without sacrificing solution fidelity.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Learning to Search
• Computer Vision: Low Level & Physics-based Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

HyperDeepONets were introduced in Lee, Cho and Hwang [ICLR, 2023] as an alternative architecture for operator learning, in which a hypernetwork generates the weights for the trunk net of a DeepONet. While this improves expressivity, it incurs high memory and computational costs due to the large number of output parameters required. In this work we introduce, in the physics-informed machine learning setting, a variation, PI-LoRA-HyperDeepONets, which leverage low-rank adaptation (LoRA) to reduce complexity by decomposing the hypernetwork's output layer weight matrix into two smaller low-rank matrices. This reduces the number of trainable parameters while introducing an extra regularization of the trunk networks' weights. Through extensive experiments on both ordinary and partial differential equations we show that PI-LoRA-HyperDeepONets achieve up to 70% reduction in parameters and consistently outperform regular HyperDeepONets in terms of predictive accuracy and generalization.

Problem

• Reducing memory and computational costs in HyperDeepONets
• Improving predictive accuracy for differential equations
• Enhancing generalization with low-rank adaptation

Innovation

• Low-rank adaptation reduces parameter count
• Hypernetwork generates trunk net weights
• Improved accuracy with fewer parameters