Early Classification of Time Series: Taxonomy and Benchmark

2024-06-26

AI Summary

Time-series early classification (ECTS) inherently faces a trade-off between accuracy and decision latency, yet existing studies lack a unified, reproducible evaluation protocol, hindering fair cross-method comparison. To address this, we propose the first principle-driven taxonomy of ECTS methodologies, formally defining standardized evaluation dimensions—accuracy, latency, and decision cost—and establishing the first large-scale, multi-faceted benchmarking framework. We open-source a unified Python experimental library integrating nine state-of-the-art algorithms, enabling exhaustive evaluation across diverse datasets and latency constraints. Our large-scale empirical analysis reveals fundamental characteristics of different method classes along the accuracy–latency Pareto frontier. This work significantly enhances comparability, reproducibility, and methodological rigor in ECTS research.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

In many situations, the measurements of a studied phenomenon are provided sequentially, and the prediction of its class needs to be made as early as possible so as not to incur too high a time penalty, but not too early and risk paying the cost of misclassification. This problem has been particularly studied in the case of time series, and is known as Early Classification of Time Series (ECTS). Although it has been the subject of a growing body of literature, there is still a lack of a systematic, shared evaluation protocol to compare the relative merits of the various existing methods. This document begins by situating these methods within a principle-based taxonomy. It defines dimensions for organizing their evaluation, and then reports the results of a very extensive set of experiments along these dimensions involving nine state-of-the art ECTS algorithms. In addition, these and other experiments can be carried out using an open-source library in which most of the existing ECTS algorithms have been implemented (see url{https://github.com/ML-EDM/ml_edm}).

Problem

• Early classification of time series to balance accuracy and timeliness
• Lack of systematic evaluation protocol for comparing ECTS methods
• Proposing a taxonomy and benchmark for ECTS algorithm performance

Innovation

• Develops taxonomy for early time series classification
• Benchmarks nine state-of-the-art ECTS algorithms
• Provides open-source library for ECTS evaluation