Evaluating Machine Learning Models for Post-Wildfire Debris-Flow Prediction

2026-08-05

AI Summary

This study addresses key challenges in post-wildfire debris flow prediction, including feature space overlap, data scarcity, and model interpretability. It systematically evaluates the performance of 15 machine learning models and introduces TabPFN to this domain for the first time, complemented by SHAP-based interpretability analysis. To mitigate limited training data, the authors propose a novel synthetic data augmentation approach leveraging TabPFN. Experimental results demonstrate that TabPFN achieves a Threat Score of 0.637 on original data, while the synthetic data significantly enhances the performance of all models except CNN, yielding an average Threat Score improvement of 0.041 for deep learning models. These findings validate the effectiveness and innovation of the proposed framework.

Abstract

Prediction of post-wildfire debris flows is critical for mitigating hazards to communities, infrastructure, and resources during intense rainfall in recently burned areas. However, identifying reliable machine learning models is complicated by overlapping debris-flow and non-debris-flow events in feature space, the need for model interpretability, and limited training data. This paper addresses these challenges through a systematic evaluation of machine learning models in terms of predictive performance, feature importance, and synthetic data augmentation. Using basin-scale observations of post-wildfire debris-flow events across the western United States, we compare 15 models, including the Tabular Prior-Data Fitted Network (TabPFN). Repeated stratified cross-validation shows that TabPFN achieves the highest unaugmented performance with a threat score of 0.637, closely followed by the best tree-based models. SHapley Additive exPlanations (SHAP) are used to identify the features driving predictions, revealing that short-duration rainfall intensity and storm accumulation consistently rank highest, while burn severity and terrain features contribute less. We further evaluate synthetic data augmentation using TabPFN-generated samples to address the scarcity of debris-flow observations. Synthetic augmentation improves the performance of all models except CNN, with the largest mean threat score increase of +0.041 among the deep learning models. By combining rigorous model benchmarking, interpretable feature analysis, and synthetic data augmentation, this work provides a comprehensive framework for improving post-wildfire debris-flow prediction.

Problem

• post-wildfire debris-flow prediction
• machine learning evaluation
• class imbalance
• model interpretability
• limited training data

Innovation

• TabPFN
• SHAP
• synthetic data augmentation
• debris-flow prediction
• machine learning benchmarking