Feature Attribution-Based Explainability Analysis of Deep Learning Models in Predictive Process Monitoring

2026-07-20

AI Summary

This work addresses the challenge of providing trustworthy explanations for deep learning models in predictive process monitoring, where their black-box nature and the limitations of existing attribution methods hinder both computational efficiency for long traces and semantic fidelity to control-flow dynamics. The authors propose a novel local post-hoc interpretability approach that leverages control-flow structures to semantically segment event logs and computes SHAP attributions over these segments to identify critical process fragments and turning points influencing predictions. Experimental results demonstrate that the method accurately captures known logical turning points on synthetic datasets and effectively uncovers the underlying dynamic mechanisms driving predictions in real-world loan approval and municipal process logs, achieving a balanced trade-off between computational efficiency and process-aware semantic interpretability.

Abstract

Predictive process monitoring supports the optimization and control of operational business processes by forecasting the future state or outcome of ongoing cases. While deep neural networks have achieved strong performance for these tasks by modeling sequential dependencies in event logs, their black-box nature limits trust and practical adoption. Feature attribution methods are often used to address this, but applying them directly poses a dilemma: event-level attributions impose high computational complexity for long traces, while explanations based on aggregated trace representations often fail to capture the underlying control-flow dynamics. To address this issue, we propose a local post-hoc explainability method for deep neural networks in outcome prediction. The method relies on a control-flow-aware segmentation algorithm that partitions a trace into meaningful segments and supports the computation of segment-level SHAP explanations. This makes it possible to identify which parts of a trace influence a prediction and which change points steer the case toward the predicted outcome. We assess the proposed segmentation method on a synthetic dataset with known process logic, where meaningful change points can be explicitly verified, and we demonstrate its usefulness on real-world event logs from a loan application process and an administrative process of a Dutch municipality.

Problem

• Predictive Process Monitoring
• Deep Learning
• Feature Attribution
• Explainability
• Control-Flow Dynamics

Innovation

• feature attribution
• control-flow-aware segmentation
• SHAP explanations
• predictive process monitoring
• post-hoc explainability