PI-MFM: Physics-informed multimodal foundation model for solving partial differential equations

2025-12-28

AI Summary

Existing PDE solvers rely heavily on large-scale labeled data and neglect underlying physical laws, limiting generalization across diverse PDE families. This work proposes a physics-informed multimodal foundation model that takes symbolic PDE expressions as input. It introduces, for the first time, a symbol-driven vectorized physics loss auto-assembly mechanism, enabling analytical residual construction and unifying pretraining and fine-tuning objectives. The model integrates symbolic differentiation with automatic differentiation, and incorporates physics-constrained pretraining, dynamic collocation point resampling, and finite-difference contrastive learning. Evaluated on a benchmark of 13 time-varying parametric PDEs, it significantly outperforms purely data-driven methods: achieving over 40% error reduction under sparse labeling, attaining ~1% test error after zero-shot physics-guided adaptation, and demonstrating markedly improved robustness to measurement noise.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Partial differential equations (PDEs) govern a wide range of physical systems, and recent multimodal foundation models have shown promise for learning PDE solution operators across diverse equation families. However, existing multi-operator learning approaches are data-hungry and neglect physics during training. Here, we propose a physics-informed multimodal foundation model (PI-MFM) framework that directly enforces governing equations during pretraining and adaptation. PI-MFM takes symbolic representations of PDEs as the input, and automatically assembles PDE residual losses from the input expression via a vectorized derivative computation. These designs enable any PDE-encoding multimodal foundation model to be trained or adapted with unified physics-informed objectives across equation families. On a benchmark of 13 parametric one-dimensional time-dependent PDE families, PI-MFM consistently outperforms purely data-driven counterparts, especially with sparse labeled spatiotemporal points, partially observed time domains, or few labeled function pairs. Physics losses further improve robustness against noise, and simple strategies such as resampling collocation points substantially improve accuracy. We also analyze the accuracy, precision, and computational cost of automatic differentiation and finite differences for derivative computation within PI-MFM. Finally, we demonstrate zero-shot physics-informed fine-tuning to unseen PDE families: starting from a physics-informed pretrained model, adapting using only PDE residuals and initial/boundary conditions, without any labeled solution data, rapidly reduces test errors to around 1% and clearly outperforms physics-only training from scratch. These results show that PI-MFM provides a practical and scalable path toward data-efficient, transferable PDE solvers.

Problem

• Develops a physics-informed multimodal foundation model for solving PDEs
• Enforces governing equations during training to reduce data dependency
• Enables zero-shot fine-tuning to unseen PDE families without labeled data

Innovation

• Physics-informed pretraining enforces governing equations directly
• Automatic PDE residual assembly via vectorized derivative computation
• Zero-shot fine-tuning to unseen PDE families without labeled data