LLMs that Understand Processes: Instruction-tuning for Semantics-Aware Process Mining

2025-08-22

AI Summary

Large language models (LLMs) exhibit limited task generalization and heavy reliance on task-specific fine-tuning in semantic-aware process mining. Method: We propose a unified multi-task instruction-tuning framework that constructs semantically enriched prompt–answer pairs covering anomaly detection, next-activity prediction, and process discovery, enabling LLMs to internalize behavioral logic and semantic constraints of business processes through joint instruction tuning. Contribution/Results: This work is the first to systematically introduce instruction tuning into semantic-aware process mining, achieving cross-task zero-shot and few-shot transfer while substantially reducing dependence on task-specific fine-tuning. Experiments demonstrate significant performance gains in process discovery and next-activity prediction; anomaly detection results vary with task composition, empirically confirming the critical impact of task selection on instruction-tuning efficacy.

Technology Category

• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Process mining is increasingly using textual information associated with events to tackle tasks such as anomaly detection and process discovery. Such semantics-aware process mining focuses on what behavior should be possible in a process (i.e., expectations), thus providing an important complement to traditional, frequency-based techniques that focus on recorded behavior (i.e., reality). Large Language Models (LLMs) provide a powerful means for tackling semantics-aware tasks. However, the best performance is so far achieved through task-specific fine-tuning, which is computationally intensive and results in models that can only handle one specific task. To overcome this lack of generalization, we use this paper to investigate the potential of instruction-tuning for semantics-aware process mining. The idea of instruction-tuning here is to expose an LLM to prompt-answer pairs for different tasks, e.g., anomaly detection and next-activity prediction, making it more familiar with process mining, thus allowing it to also perform better at unseen tasks, such as process discovery. Our findings demonstrate a varied impact of instruction-tuning: while performance considerably improved on process discovery and prediction tasks, it varies across models on anomaly detection tasks, highlighting that the selection of tasks for instruction-tuning is critical to achieving desired outcomes.

Problem

• Instruction-tuning LLMs for generalization in process mining tasks
• Overcoming task-specific limitations in semantics-aware process mining
• Evaluating instruction-tuning impact on anomaly detection and prediction performance

Innovation

• Instruction-tuning for semantics-aware process mining
• Using prompt-answer pairs across multiple tasks
• Improves generalization for unseen process tasks