🤖 AI Summary
This work addresses the challenge of efficiently adapting pre-trained foundation models for partial differential equations (PDEs) under data scarcity and distribution shift. We propose a physics-informed fine-tuning framework that incorporates physical constraints—such as PDE residuals and boundary conditions—directly into the fine-tuning objective, enabling data-efficient adaptation without requiring ground-truth solutions. To our knowledge, this is the first systematic demonstration of the effectiveness of physics-informed fine-tuning for transferring PDE foundation models. By integrating a hybrid fine-tuning strategy, our approach significantly enhances out-of-distribution generalization. Experiments show that, even in the absence of ground-truth solutions, our method achieves accuracy comparable to purely data-driven approaches on unseen PDE tasks and consistently outperforms them when only limited data are available.
📝 Abstract
Foundation models for partial differential equations (PDEs) have emerged as powerful surrogates pre-trained on diverse physical systems, but adapting them to new downstream tasks remains challenging due to limited task-specific data and distribution shifts. While fine-tuning has proven transformative in natural language processing, best practices for adapting PDE foundation models remain underexplored. Although physics-informed training has successfully trained accurate solvers across a wide range of PDE problems, its potential for fine-tuning data-based foundation models has not been systematically studied. In this work, we introduce a physics-informed fine-tuning framework that adapts pre-trained PDE foundation models by incorporating physical constraints (PDE residuals and boundary conditions) directly into the fine-tuning objective. This enables effective adaptation in data-scarce regimes while promoting physical consistency. We evaluate our method on a downstream task composed of an unseen PDE class and compare it with data-driven finetuning counterparts. Our results demonstrate that physics-informed fine-tuning achieves competitive accuracy without requiring PDE solutions for training. Furthermore, a hybrid fine-tuning strategy yields superior generalization to out-of-distribution scenarios when only minimal training data is available. These findings establish physics-informed fine-tuning as a scalable and data-efficient paradigm, providing a physically interpretable pathway for adapting foundation models in scientific machine learning.