validate simulation models

In the pre-prototype phase of complex novel systems, the absence of empirical data impedes rigorous assessment of simulation model credibility. Method: This paper proposes a physics-fidelity-based model trust evaluation method that bypasses reliance on real-world measurements. Instead, it quantifies model applicability by systematically analyzing the completeness of represented physical phenomena, the mathematical complexity of their formulation, and the fidelity of emergent behavior modeling. Contribution/Results: The approach enables objective, quantitative ranking of multiple candidate models under data-scarce conditions—thereby significantly enhancing the reliability of simulation-driven decisions during early-stage design. It establishes both theoretical foundations and practical tools for model-based design in high-uncertainty scenarios, advancing trustworthy digital twin development and physics-informed simulation validation.

Model fidelity—the degree of correspondence between simulation and reality—lacks a formal, axiomatic foundation in digital engineering, resulting in ambiguous evaluation criteria and poor cross-domain comparability. Method: This paper introduces the first rigorous, verifiable theoretical framework for fidelity assessment, grounded in seven foundational axioms encompassing consistency, measurability, scale invariance, and other essential properties; the framework enables formal verification and comparative analysis of fidelity metrics. Empirical validation is conducted via integration into ground-vehicle modeling, demonstrating feasibility and practical guidance within existing evaluation paradigms. Contribution/Results: The work fills a critical theoretical gap in fidelity science and establishes a universal, standards-ready paradigm for fidelity assessment—directly advancing digital twin development, simulation verification and validation (V&V), and model-based systems engineering. It further provides a clear, principled roadmap for future methodological evolution and standardization.

This study addresses the challenge of effectively validating input model specifications in digital twin simulations, where conventional approaches—relying solely on marginal output distributions—often fail to detect misspecified joint input models. To overcome this limitation, the authors propose a novel statistical validation framework based on sub-trajectory conditioning. By repeatedly restarting simulations from observed system states while conditioning on subsets of random inputs, the method constructs conditional output distributions that enable goodness-of-fit testing of the full joint input model. This approach innovatively transcends the constraints of marginal validation and is complemented by diagnostic tools to pinpoint specific input sources responsible for detected discrepancies. Empirical evaluations on M/M/1 and tandem queueing systems demonstrate the framework’s heightened sensitivity and effectiveness, successfully identifying input model misspecifications that traditional methods overlook.

In model-based systems engineering, low experimental data reuse efficiency and excessive redundant experiments hinder digital engineering agility. To address this, this paper proposes a case-based reasoning (CBR)-driven experimental management framework that explicitly integrates domain knowledge. The framework features structured experimental metadata modeling, digital twin–enabled scenario semantic alignment, and an interpretable similarity assessment mechanism to intelligently determine whether historical experiments can be transferred to address new verification queries. Its key innovation lies in embedding domain knowledge explicitly into both the CBR retrieval and adaptation stages, thereby enabling trustworthy cross-operating-condition and cross-configuration experimental data reuse. Evaluated on an industrial-scale vehicle energy system design case, the framework reduces redundant experiments by 37% and shortens early verification cycles by 42% on average, significantly enhancing iterative efficiency in digital engineering and advancing intelligent experimental management.

论文提出基于风险的框架，通过验证、确认及敏感性分析等手段，提高自动驾驶系统虚拟测试仿真结果的可信度。

Existing Simulink model checkers often produce verification results inconsistent with simulation outcomes due to the absence of bit-precise formal semantics for modeling elements and numerical behaviors, undermining their reliability. This work proposes the first bit-precise conformance testing methodology tailored for Simulink model checkers. By formally specifying the semantics of fundamental blocks, constructing a test suite covering ten block categories, and integrating SMT solving within an automated framework, the approach systematically evaluates behavioral alignment across tools. Experimental results demonstrate that the method effectively uncovers inconsistencies: while the third-party checker SmtMC passes all tests, Simulink Design Verifier exhibits only 94–96% conformance with the simulator, and its agreement with other checkers drops further to 80–90%. The framework also precisely identifies the root causes of these discrepancies.

This study addresses a critical limitation in existing simulation credibility assessment approaches, which predominantly focus on individual models and thus fail to capture the reliability of complex, multi-model architectures. Moving beyond the single-model evaluation paradigm, this work redefines trustworthiness at the architectural level and proposes a multidimensional framework that integrates sensitivity analysis, expert knowledge, explainable artificial intelligence, and complex network modeling. Through a systematic comparison of diverse methodologies across dimensions such as methodological rigor, generalizability, and computational resource demands, the research offers both theoretical foundations and practical guidance for constructing high-assurance simulation architectures.

This study addresses the limitation of existing generative model benchmarks, which focus solely on code compilation or structural similarity and fail to verify engineering compliance. We construct a text-to-executable Simulink model generation benchmark spanning ten domains and propose an executable system archive-based method for aligning with engineering specifications. Furthermore, we design a hierarchical automated evaluation mechanism alongside a six-dimensional dynamic response metric system, enabling end-to-end assessment from deliverability and executability to engineering qualification. Experimental results demonstrate that the best-performing agent achieves a score of only 42.86, confirming a significant disconnect between structural similarity and engineering performance. These findings reveal critical capability bottlenecks in current large language models regarding the generation of qualified systems that satisfy multidimensional engineering requirements.

This study investigates the use of large language models (LLMs) to automatically translate neutral graph representations of fluid systems into high-quality, functionally correct code executable in mainstream simulation environments such as WNTR and Modelica. The authors systematically evaluate ten state-of-the-art LLMs combined with six prompting strategies across multiple benchmark scenarios, assessing generated code through software quality metrics and simulation fidelity. This work presents the first systematic comparison in the domain of fluid system modeling that examines how different LLMs and prompt engineering techniques influence both syntactic correctness and functional fidelity of generated simulation code, offering empirical guidance for model-driven code generation. Experimental results demonstrate that optimal configurations can produce syntactically valid code; however, a significant gap remains in achieving high simulation fidelity, highlighting key directions for future improvement.

本文提出两种技术，通过增加推理时间和模拟真实部署环境来提高对齐评估的真实性，解决模型在测试与实际部署中表现差异的问题。