On the Simplification of Neural Network Architectures for Predictive Process Monitoring

2025-09-21

AI Summary

Predictive Process Monitoring (PPM) faces deployment challenges due to its reliance on computationally expensive deep models (e.g., LSTM, Transformer). This work systematically investigates the impact of neural architecture simplification on PPM performance, quantifying the accuracy–efficiency trade-off under parameter reduction and depth pruning. Experiments across five real-world event logs demonstrate that Transformers tolerate up to 85% parameter compression with only a 2–3% drop in prediction accuracy, whereas LSTMs exhibit pronounced degradation—particularly for remaining time prediction—under similar compression. The study reveals substantial model redundancy inherent in PPM tasks and provides the first empirical validation that lightweight Transformers maintain high predictive fidelity while significantly improving deployment feasibility. These findings establish a new architectural design paradigm and an empirical benchmark for efficient, scalable process prediction.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Learning & Optimization for NLP
• Data Mining & Knowledge Management: Data Compression

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Predictive Process Monitoring (PPM) aims to forecast the future behavior of ongoing process instances using historical event data, enabling proactive decision-making. While recent advances rely heavily on deep learning models such as LSTMs and Transformers, their high computational cost hinders practical adoption. Prior work has explored data reduction techniques and alternative feature encodings, but the effect of simplifying model architectures themselves remains underexplored. In this paper, we analyze how reducing model complexity, both in terms of parameter count and architectural depth, impacts predictive performance, using two established PPM approaches. Across five diverse event logs, we show that shrinking the Transformer model by 85% results in only a 2-3% drop in performance across various PPM tasks, while the LSTM proves slightly more sensitive, particularly for waiting time prediction. Overall, our findings suggest that substantial model simplification can preserve predictive accuracy, paving the way for more efficient and scalable PPM solutions.

Problem

• Simplifies neural network architectures for predictive process monitoring
• Analyzes impact of reduced model complexity on predictive performance
• Explores trade-offs between computational efficiency and accuracy preservation

Innovation

• Simplifies Transformer model by 85% size reduction
• Evaluates impact of reduced parameter count and depth
• Shows minimal performance loss with major complexity decrease