design simulation studies

In model-based systems engineering, low experimental data reuse efficiency and excessive redundant experiments hinder digital engineering agility. To address this, this paper proposes a case-based reasoning (CBR)-driven experimental management framework that explicitly integrates domain knowledge. The framework features structured experimental metadata modeling, digital twin–enabled scenario semantic alignment, and an interpretable similarity assessment mechanism to intelligently determine whether historical experiments can be transferred to address new verification queries. Its key innovation lies in embedding domain knowledge explicitly into both the CBR retrieval and adaptation stages, thereby enabling trustworthy cross-operating-condition and cross-configuration experimental data reuse. Evaluated on an industrial-scale vehicle energy system design case, the framework reduces redundant experiments by 37% and shortens early verification cycles by 42% on average, significantly enhancing iterative efficiency in digital engineering and advancing intelligent experimental management.

This study addresses the challenges of control design in complex industrial processes characterized by multivariable coupled dynamics by proposing an automated control strategy generation framework that integrates large language models (LLMs) with Bayesian optimization. The approach decomposes control design into structured code generation steps, ensuring physical consistency through execution-based validation and feedback-driven repair. It pioneers the automatic synthesis of decentralized PI controller architectures and their tuning environments directly from dynamic process models. Evaluated on a nonlinear gas preheater benchmark, the generated control schemes—subsequently refined via Bayesian optimization—achieve a 26.5% improvement in closed-loop performance and significantly enhance the transient response of pressure loops, thereby demonstrating the method’s effectiveness and novelty.

In A/B testing, control variates and regression adjustment are widely used variance reduction techniques, yet their theoretical relationship remains unclear, their methodological frameworks are disjointed, and both have long been confined to design-driven paradigms. Method: This paper establishes, for the first time, a formal equivalence between these two approaches and proposes a novel grouped coefficient estimation method that unifies design-based and model-based estimation frameworks—enabling a paradigm shift from design-driven to model-driven inference. Contribution/Results: Theoretical analysis demonstrates improved estimation accuracy and statistical power. Empirical validation on millions of real-world experiments at ByteDance confirms efficacy: the proposed method has been fully deployed in its online experimentation platform, yielding an average 12.3% increase in statistical significance and a 19.6% improvement in detection sensitivity.

This study addresses the challenge of effectively validating input model specifications in digital twin simulations, where conventional approaches—relying solely on marginal output distributions—often fail to detect misspecified joint input models. To overcome this limitation, the authors propose a novel statistical validation framework based on sub-trajectory conditioning. By repeatedly restarting simulations from observed system states while conditioning on subsets of random inputs, the method constructs conditional output distributions that enable goodness-of-fit testing of the full joint input model. This approach innovatively transcends the constraints of marginal validation and is complemented by diagnostic tools to pinpoint specific input sources responsible for detected discrepancies. Empirical evaluations on M/M/1 and tandem queueing systems demonstrate the framework’s heightened sensitivity and effectiveness, successfully identifying input model misspecifications that traditional methods overlook.

This work addresses the testing challenge of high-dimensional, non-deterministic, and computationally expensive software systems—exemplified by the CARLA autonomous driving simulator—where latent variables and variable interactions undermine causal inference. Existing causal testing methods assume full observability and absence of interactions, rendering them inapplicable to realistic, partially observable settings. To overcome this, we introduce effect modification analysis and instrumental variable methods into software causal testing for the first time, establishing a robust verification framework capable of modeling latent variables and identifying interaction effects. Crucially, our approach requires neither full log recording nor source-code instrumentation; it achieves reliable validation of three system-level requirements in CARLA using only limited, controlled data under low observability. As a result, it substantially reduces dependence on large-scale test data and strong observability assumptions, advancing practical causal testing for complex cyber-physical systems.

This work proposes the first general framework to systematically quantify and apportion epistemic uncertainty arising from substituting true subprocesses with approximate or learned submodels in stochastic simulation and digital twin applications. The framework constructs confidence or credible intervals for performance metrics via bootstrapping and Bayesian model averaging, and employs a tree-based decomposition to allocate total output variability to individual submodels, yielding importance scores. It is compatible with both parametric and nonparametric models, supports frequentist and Bayesian paradigms, and accommodates dynamic initialization scenarios. Validation on synthetic data and a call center digital twin demonstrates that the method effectively reveals each submodel’s contribution to overall uncertainty, significantly enhancing the interpretability and reliability of simulation outcomes.

This work addresses the behavioral gap between formal verification and actual execution in traditional engineering approaches, which often neglect execution semantics. To bridge this semantic divide, the paper proposes a Modeling and Simulation-Based Engineering (MSBE) methodology that explicitly treats execution semantics as a first-class engineering entity. It defines executability as the admissible model space induced by the stabilization of execution conditions and unifies model behavior with physical execution through an iterative cycle of formal execution, experimental execution, verification, and activity-mediated validation. Integrating formal methods, simulation-based verification, activity theory, and constraint modeling, MSBE establishes a general-purpose engineering framework applicable to diverse cyber-physical systems (CPS). The approach demonstrates its generality and effectiveness across four CPS categories: human-centric, biophysical, technological, and digital twin systems.

This work proposes a novel approach to black-box testing of Functional Mock-up Units (FMUs) by integrating large language models (LLMs) with a human-in-the-loop mechanism. Addressing the inefficiency and poor interpretability of traditional FMU-based dynamic simulation testing—which relies on manually crafted scenarios—the method automatically generates structured Given-When-Then test objectives from FMU interface and functional specifications, and constructs complete test plans comprising input sequences and assertion oracles. Upon simulation execution, the framework produces visualizable logs and statistical evaluation metrics. The approach significantly enhances test design efficiency and result interpretability, facilitates test asset reuse, and demonstrates effectiveness on a lubricating oil cooling system by autonomously generating executable test scenarios and delivering objective-level pass-rate analysis.

This study addresses the lack of a systematic framework for identifying critical input variables and conducting sensitivity analysis under uncertainty in complex simulations, particularly in military decision-making contexts. The authors propose a unified sensitivity analysis framework that integrates local and global methods—including variance-based, derivative-based, screening, and uncertainty quantification techniques—and strategically maps these approaches to specific decision objectives such as factor prioritization, fixing, variance reduction, and mapping. Innovatively, the framework introduces a “sensitivity audit” mechanism to enhance traceability of model assumptions and promote responsible model usage. By providing a structured guide for high-dimensional, complex simulation systems, this work significantly improves model interpretability, transparency, and the credibility of decisions derived from such models.

This study addresses the persistent gap between theoretical control performance and its practical realization in real-world robotic systems, often caused by inadequate discretization, insufficient real-time guarantees, and weak error handling in control software. For the first time from a software engineering perspective, the authors systematically analyze 184 open-source robotic controllers through code review, empirical analysis, and test evaluation, uncovering common deficiencies in application scenarios, implementation details, and verification practices. The findings reveal that most implementations fail to properly account for critical system constraints, and their testing strategies inadequately validate the theoretical assurances they claim. This work highlights a significant disconnect between implementation quality and theoretical promises, offering concrete directions and practical guidelines for developing reliable, verifiable robotic control software.