evidence-guided architecture selection

This work addresses the challenge of implicit constraints—manifested as evaluation failures—arising from unreliable physics-based simulations in system architecture optimization. To tackle this, we propose a surrogate modeling framework that integrates probabilistic feasibility prediction with Bayesian optimization. Methodologically, we introduce a novel hybrid discrete Gaussian process to model the Probability of Validity (PoV), coupled with an interior-point selection strategy based on a minimum PoV threshold; the framework natively supports hierarchical design variables and multi-objective optimization. Our approach achieves the first successful solution for a jet engine architecture optimization task with a 50% simulation failure rate. Across multiple synthetic benchmarks and real-world case studies, it significantly improves convergence robustness and optimization success rate. The implementation is publicly available as the SBArchOpt Python library.

In model-based systems engineering, low experimental data reuse efficiency and excessive redundant experiments hinder digital engineering agility. To address this, this paper proposes a case-based reasoning (CBR)-driven experimental management framework that explicitly integrates domain knowledge. The framework features structured experimental metadata modeling, digital twin–enabled scenario semantic alignment, and an interpretable similarity assessment mechanism to intelligently determine whether historical experiments can be transferred to address new verification queries. Its key innovation lies in embedding domain knowledge explicitly into both the CBR retrieval and adaptation stages, thereby enabling trustworthy cross-operating-condition and cross-configuration experimental data reuse. Evaluated on an industrial-scale vehicle energy system design case, the framework reduces redundant experiments by 37% and shortens early verification cycles by 42% on average, significantly enhancing iterative efficiency in digital engineering and advancing intelligent experimental management.

Complex system simulation faces two major challenges: high computational cost and limited interpretability of black-box models. This paper proposes a surrogate-model-driven eXplainable Artificial Intelligence (XAI) workflow that unifies global sensitivity analysis, uncertainty quantification, and local attribution methods to jointly model continuous and categorical variables. A novel explanation consistency evaluation mechanism is introduced to dynamically diagnose surrogate model adequacy, thereby guiding data acquisition optimization and structural refinement. Leveraging experimental design, a compact training dataset is constructed to train lightweight surrogates, enabling second-scale exploration of large-scale simulations. The method reveals nonlinear interactions and emergent behaviors, identifies critical design or policy levers, and pinpoints model weaknesses. Its effectiveness and generalizability are validated across diverse domains, including engineering design and socio-environmental simulation.

This work addresses Bayesian optimal experimental design under computationally expensive models with limited design evaluations. It proposes an adaptive sequential elimination algorithm that significantly reduces the variance and computational cost of nested Monte Carlo estimators by reusing parameter samples, employing common random numbers, and applying Rao–Blackwellization. A bootstrap-based probabilistic comparison mechanism is integrated to iteratively eliminate inferior designs. The method achieves high reliability while drastically reducing the number of model evaluations, making it well-suited for large-scale engineering applications where computational efficiency and decision accuracy must be carefully balanced.

Simulation optimization involving mixed-variable design spaces—comprising continuous, integer, and categorical variables—with hierarchical dependencies, conditional activation, and tree-structured relationships poses significant modeling and search challenges. Method: We propose a unified modeling paradigm that introduces *meta-variables* and *partially ordered conditional variables*, formalized via a *design space graph* to explicitly encode hierarchical and conditional variable relationships. Integrating graph-theoretic representations with feature-based modeling, we design a hierarchical kernel function and a non-Euclidean distance metric tailored to non-flat, structured domains, embedded within a surrogate-modeling framework compatible with Bayesian optimization. Contribution/Results: This is the first approach to enable unified modeling and optimization over complex conditional design spaces. Implemented in the open-source SMT 2.0 toolkit, it demonstrates substantial improvements in both modeling fidelity and search efficiency, as validated on green aircraft architecture optimization.

This work addresses the challenge of discovering mechanistic causal world models capable of answering “what if” questions in a data-efficient manner from limited interventional experiments. It proposes a novel framework that integrates large language models (LLMs) as structural proposers within a Bayesian experimental design paradigm. Operating under the M-open setting, the approach iteratively expands and validates candidate models through sequential Monte Carlo, simulation-based inference, and value-of-information-driven active experimentation, enabling adaptive enrichment of the hypothesis space. Evaluated on benchmarks spanning physics, chemistry, and neural electrophysiology, the method demonstrates substantially improved data efficiency and reliability in interventional prediction, achieving state-of-the-art performance.

This study addresses the prevailing gap in AI education, which emphasizes model development while neglecting system engineering practices, leaving students ill-equipped to handle real-world challenges such as architectural design, deployment, and monitoring. To bridge this gap, the authors implemented a master’s-level course in which students built a movie recommendation system under realistic constraints, with a focus on integrating AI components into robust software systems, adopting data-driven machine learning practices, and cultivating systems-level thinking. Using a mixed-methods approach—combining analysis of student project artifacts with survey data—the research evaluates learners’ performance in architectural decision-making, integration of heterogeneous models, and adaptation to evolving requirements. Findings reveal common difficulties students encounter in AI system engineering and demonstrate the course’s effectiveness in addressing critical deficiencies in AI engineering education and enhancing systems-aware competencies.

This study addresses a critical limitation in existing simulation credibility assessment approaches, which predominantly focus on individual models and thus fail to capture the reliability of complex, multi-model architectures. Moving beyond the single-model evaluation paradigm, this work redefines trustworthiness at the architectural level and proposes a multidimensional framework that integrates sensitivity analysis, expert knowledge, explainable artificial intelligence, and complex network modeling. Through a systematic comparison of diverse methodologies across dimensions such as methodological rigor, generalizability, and computational resource demands, the research offers both theoretical foundations and practical guidance for constructing high-assurance simulation architectures.

This study addresses the current lack of human-centered, interpretable, and responsible evaluation criteria for AI in modeling and simulation. The authors propose the first multidimensional benchmark framework specifically designed to assess large language models (LLMs) through a human-centric lens, leveraging an open-source system dynamics AI platform to systematically evaluate performance across qualitative modeling, quantitative modeling, and model discussion tasks—emphasizing human-AI collaboration rather than replacement. The framework incorporates critical capabilities such as causal reasoning, iterative model refinement, and behavioral explanation, while embedding ethical and accountability considerations. Empirical results indicate that existing AI tools perform relatively well in qualitative tasks and model discussions but remain limited in causal reasoning and quantitative error correction; furthermore, different LLMs exhibit distinct strengths, with no single model emerging as universally superior.