🤖 AI Summary
Theorem provers exhibit unstable performance, poor scalability, and low reproducibility in behavioral verification—particularly for real-time and safety-critical software. Method: We systematically reproduce and extend existing benchmarks to construct an empirical evaluation framework featuring diverse behavioral models and logical specifications, enabling rigorous assessment of robustness, scalability, and reproducibility across mainstream theorem provers. Contribution/Results: Our study is the first to empirically establish strong correlations between irregular solver performance and structural problem characteristics—specifically temporal constraint density and state-space distribution. Leveraging these insights, we propose adaptive heuristic strategies and a self-optimizing solver architecture. The approach delivers measurable stability improvements for just-in-time verification in CI/CD pipelines and AI-augmented IDEs, significantly enhancing the practicality and trustworthiness of automated logical verification in high-assurance software development.
📝 Abstract
This study empirically validates automated logical specification methods for behavioural models, focusing on their robustness, scalability, and reproducibility. By the systematic reproduction and extension of prior results, we confirm key trends, while identifying performance irregularities that suggest the need for adaptive heuristics in automated reasoning. Our findings highlight that theorem provers exhibit varying efficiency across problem structures, with implications for real-time verification in CI/CD pipelines and AI-driven IDEs supporting on-the-fly validation. Addressing these inefficiencies through self-optimising solvers could enhance the stability of automated reasoning, particularly in safety-critical software verification.