A Comparison for Non-Specialists of Workflow Steps and Similarity of Factor Rankings for Several Global Sensitivity Analysis Methods

2025-10-24

AI Summary

This study addresses non-expert users by systematically evaluating the workflow feasibility and factor ranking consistency of multiple global sensitivity analysis (GSA) methods across simulation models of varying complexity. Method: It integrates Sobol’ first-order and total-effect indices with regression tree analysis, and—novelty—employs Kendall’s W to quantify inter-method ranking similarity; special attention is given to how parameter range specification affects result robustness. Contribution/Results: (1) Major GSA methods exhibit high consistency in factor importance ranking; (2) Sobol’ indices offer both interpretability and information richness, while regression trees effectively detect interaction effects; (3) Parameter range specification is identified as a critical practical determinant of GSA reliability. Collectively, these findings significantly enhance operationality and methodological rationality for non-experts in tasks such as factor screening, freezing, and prioritization.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Evaluation and Analysis
• Reasoning under Uncertainty: Causality

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Global sensitivity analysis (GSA) is a recommended step in the use of computer simulation models. GSA quantifies the relative importance of model inputs on outputs (Factor Ranking), identifies inputs that could be fixed, thus simplifying model calibration (Factor Fixing), and pinpointing areas for future data collection (Factor Prioritization). Given the wide variety of GSA methods, choosing between methods can be challenging for non-GSA experts. Issues include workflow steps and complexity, interpretation of GSA outputs, and the degree of similarity between methods in Factor Ranking. We conducted a study of both widely and less commonly used GSA methods applied to three simulators of differing complexity. All methods share common issues around implementation with specification of parameter ranges particularly critical. Similarities in Factor Rankings were generally high based on Kendall's W. Sobol' first order and total sensitivity indices were easy to interpret and informative with regression trees providing additional insight into interactions.

Problem

• Compares workflow steps and factor ranking similarity across GSA methods
• Addresses method selection challenges for non-specialists in sensitivity analysis
• Evaluates implementation issues and interpretation of sensitivity indices

Innovation

• Compares workflow steps of sensitivity analysis methods
• Evaluates factor ranking similarity using Kendall's W
• Uses Sobol indices and regression trees for interpretation