🤖 AI Summary
This work addresses the high computational cost and poor scalability of maximal frequent episode mining (MaxFEM) under low support thresholds or with long episodes. To overcome these limitations, we propose ParMaxFEM—an algorithm that introduces the first efficient parallelization of MaxFEM, implemented in C++ with multithreaded optimizations and seamlessly integrated into Desbordante, an open-source data profiling platform that treats patterns as first-class citizens for interactive analysis. Experimental results demonstrate that our single-threaded implementation achieves up to 8× speedup over the SPMF baseline, while the 8-core parallel variant attains a maximum acceleration of 35×, substantially enhancing the efficiency and practicality of mining frequent patterns in large-scale sequential data.
📝 Abstract
Episode mining aims to extract subsequences of events that possess certain distinctive properties and constitute facts valuable to the user. Maximal frequent episode mining concentrates on discovery of frequently-appearing subsequences, which are not included into any other larger frequent subsequence. The state-of-the-art for this problem is the MaxFEM algorithm which enumerates possible subsequences, while applying various pruning techniques to accelerate the search. However, this is a computationally-intensive problem: reducing the minimum number of required subsequence occurrences or increasing the length of the subsequence both substantially raise running time, which limits practical use of MaxFEM. In this paper we describe our efforts in designing a high-performing algorithm for this problem. For this we: 1) develop an efficient C++ implementation of MaxFEM, and 2) devise an efficient technique to parallelizing it. As the result, we propose an improved parallel MaxFEM variant, which we call ParMaxFEM. Additionally, we integrate the improved algorithm into Desbordante - a high-performance, open-source data profiler with deep Python integration that treats patterns as first-class entities and allows users to develop their custom programs that can include discovery and validation of patterns. To evaluate our approach we compare both C++ implementations with the original SPMF implementation. Experiments demonstrated that our reimplemented version provides up to $8\times$ speedup over the SPMF baseline, while our parallelization technique provides up to $35\times$ improvement overall (on 8 cores).