Formal verification of tree-based machine learning models for lateral spreading

2026-03-17

AI Summary

This work addresses the critical issue that existing tree-based models in geotechnical hazard prediction may violate fundamental physical laws and lack global consistency guarantees. For the first time in geotechnical engineering, formal verification is integrated into machine learning by encoding tree ensemble models—such as XGBoost and Explainable Boosting Machines (EBM)—into Satisfiability Modulo Theories (SMT) logical formulas. This enables rigorous verification across the entire input domain against four predefined physical constraints, establishing a verification-correction feedback loop. Experiments reveal that an unconstrained EBM violates all four physical specifications; while incorporating constraints enforces compliance with three of them, it incurs a drop in predictive accuracy. These findings highlight an inherent trade-off between model accuracy and physical consistency, and underscore the limitations of post-hoc interpretability methods in ensuring physically plausible predictions.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Machine learning models for geotechnical hazard prediction can achieve high accuracy while learning physically inconsistent relationships from sparse or biased training data. Current remedies (post-hoc explainability, such as SHAP and LIME, and training-time constraints) either diagnose individual predictions approximately or restrict model capacity without providing exhaustive guarantees. This paper encodes trained tree ensembles as logical formulas in a Satisfiability Modulo Theories (SMT) solver and checks physical specifications across the entire input domain, not just sampled points. Four geotechnical specifications (water table depth, PGA monotonicity, distance safety, and flat-ground safety) are formalized as decidable logical formulas and verified via SMT against both XGBoost ensembles and Explainable Boosting Machines (EBMs) trained on the 2011 Christchurch earthquake lateral spreading dataset (7,291 sites, four features). The SMT solver either produces a concrete counterexample where a specification fails or proves that no violation exists. The unconstrained EBM (80.1% accuracy) violates all four specifications. A fully constrained EBM (67.2%) satisfies three of four specifications, demonstrating that iterative constraint application guided by verification can progressively improve physical consistency. A Pareto analysis of 33 model variants reveals a persistent trade-off, as none of the variants studied achieve both greater than 80% accuracy and full compliance with the specified set. SHAP analysis of specification counterexamples shows that the offending feature can rank last, demonstrating that post-hoc explanations do not substitute for formal verification. These results establish a verify-fix-verify engineering loop and a formal certification for deploying physically consistent ML models in safety-critical geotechnical applications.

Problem

• formal verification
• physical consistency
• tree-based machine learning
• geotechnical hazard prediction
• SMT

Innovation

• formal verification
• SMT solving
• tree ensemble
• physical consistency
• geotechnical hazard prediction