Quantifying model prediction sensitivity to model-form uncertainty

2025-09-10

AI Summary

Model-form uncertainty (MFU)—arising from simplifying modeling assumptions and particularly challenging to quantify during extrapolation—remains a critical, yet poorly addressed, source of epistemic uncertainty in physics-based modeling. Existing approaches heavily rely on calibration data and cannot isolate the independent influence of individual assumptions on predictions. Method: We propose a calibration-free MFU quantification framework that parameterizes modeling assumptions and integrates grouped variance-based sensitivity analysis to explicitly characterize how assumption changes propagate into predictive variance. The method accommodates parameter dependencies and enables assumption importance ranking under extrapolative conditions. Contribution/Results: Experiments demonstrate that our approach effectively identifies the assumptions dominating prediction uncertainty. It provides quantitative guidance for model simplification, verification, and refinement, thereby significantly enhancing the credibility and robustness of complex physics-based models.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Model-form uncertainty (MFU) in assumptions made during physics-based model development is widely considered a significant source of uncertainty; however, there are limited approaches that can quantify MFU in predictions extrapolating beyond available data. As a result, it is challenging to know how important MFU is in practice, especially relative to other sources of uncertainty in a model, making it difficult to prioritize resources and efforts to drive down error in model predictions. To address these challenges, we present a novel method to quantify the importance of uncertainties associated with model assumptions. We combine parameterized modifications to assumptions (called MFU representations) with grouped variance-based sensitivity analysis to measure the importance of assumptions. We demonstrate how, in contrast to existing methods addressing MFU, our approach can be applied without access to calibration data. However, if calibration data is available, we demonstrate how it can be used to inform the MFU representation, and how variance-based sensitivity analysis can be meaningfully applied even in the presence of dependence between parameters (a common byproduct of calibration).

Problem

• Quantifying model-form uncertainty in physics-based predictions
• Measuring importance of model assumptions without calibration data
• Addressing dependence between parameters in sensitivity analysis

Innovation

• Parameterized modifications to model assumptions
• Grouped variance-based sensitivity analysis
• Applied without requiring calibration data