Quantifying and Attributing Submodel Uncertainty in Stochastic Simulation Models and Digital Twins

2026-02-17

AI Summary

This work proposes the first general framework to systematically quantify and apportion epistemic uncertainty arising from substituting true subprocesses with approximate or learned submodels in stochastic simulation and digital twin applications. The framework constructs confidence or credible intervals for performance metrics via bootstrapping and Bayesian model averaging, and employs a tree-based decomposition to allocate total output variability to individual submodels, yielding importance scores. It is compatible with both parametric and nonparametric models, supports frequentist and Bayesian paradigms, and accommodates dynamic initialization scenarios. Validation on synthetic data and a call center digital twin demonstrates that the method effectively reveals each submodel’s contribution to overall uncertainty, significantly enhancing the interpretability and reliability of simulation outcomes.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

Stochastic simulation is widely used to study complex systems composed of various interconnected subprocesses, such as input processes, routing and control logic, optimization routines, and data-driven decision modules. In practice, these subprocesses may be inherently unknown or too computationally intensive to directly embed in the simulation model. Replacing these elements with estimated or learned approximations introduces a form of epistemic uncertainty that we refer to as submodel uncertainty. This paper investigates how submodel uncertainty affects the estimation of system performance metrics. We develop a framework for quantifying submodel uncertainty in stochastic simulation models and extend the framework to digital-twin settings, where simulation experiments are repeatedly conducted with the model initialized from observed system states. Building on approaches from input uncertainty analysis, we leverage bootstrapping and Bayesian model averaging to construct quantile-based confidence or credible intervals for key performance indicators. We propose a tree-based method that decomposes total output variability and attributes uncertainty to individual submodels in the form of importance scores. The proposed framework is model-agnostic and accommodates both parametric and nonparametric submodels under frequentist and Bayesian modeling paradigms. A synthetic numerical experiment and a more realistic digital-twin simulation of a contact center illustrate the importance of understanding how and how much individual submodels contribute to overall uncertainty.

Problem

• submodel uncertainty
• stochastic simulation
• digital twins
• epistemic uncertainty
• output variability

Innovation

• submodel uncertainty
• stochastic simulation
• digital twin
• uncertainty quantification
• Bayesian model averaging