Risk Is Not the Target: A Monotonic Framework for Evaluating Wildfire Operational Risk Signals

2026-04-23

AI Summary

Traditional wildfire risk assessments rely heavily on classification metrics such as F1 score and Intersection over Union (IoU), often overlooking the monotonic relationship between risk scores and operational burdens—such as actual fire counts and resource allocation. This work proposes a novel monotonicity-based evaluation framework that shifts the focus of risk model assessment from mere predictive accuracy to explanatory power regarding operational dynamics. The framework is applied to compare the expert-derived DFE index, a GRU-based temporal model, and the multi-agent system FARS in the Alpes-Maritimes region of France. Results reveal that while DFE exhibits weaker classification performance, it achieves the strongest global monotonicity; GRU demonstrates strong local monotonicity but uneven risk distribution; and FARS exposes structural deficiencies in upstream signal processing. These findings validate the efficacy of an evaluation paradigm centered on operational dynamics.

Abstract

Evaluating wildfire risk systems using standard machine-learning metrics such as F1-score or IoU is fundamentally flawed: these metrics assess event prediction accuracy, not the operational coherence of a continuous risk signal. This work proposes a novel monotonic evaluation framework that measures whether increases in a predicted risk score consistently correspond to increases in observed operational load, such as number of fires, intervention time, and deployed resources. Moreover, we compare three structurally different approaches on the French Alpes-Maritimes department: the expert-based DFE index, GRU- based predictive models, and FARS, a hybrid multi-agent system combining predictive AI with LLM-based reasoning. Experimental results reveal that the DFE, despite poor classification metrics, exhibits the most balanced monotonic behavior across the full risk scale. GRU models achieve strong local monotonicity but fail to produce well-distributed risk levels. FARS inherits and reveals the structural limitations of upstream signals rather than correcting them. The central finding is a paradigm shift: a good risk model does not predict fires accurately, but one whose ordinal scale meaningfully explains operational dynamics, as proved in this paper. Code of the monotonic framework is available on github.

Problem

• wildfire risk
• operational coherence
• risk evaluation
• monotonicity
• machine learning metrics

Innovation

• monotonic evaluation
• wildfire risk signals
• operational coherence
• ordinal risk scale
• hybrid multi-agent system