Are We Really Benchmarking Forecasting Models? The Impact of Preprocessing on Time Series Performance

2026-10-06

AI Summary

This study addresses the oversight of data preprocessing in existing time series benchmarks, which introduces structural biases and obscures the potential of simple models. To this end, we propose the first preprocessing-aware evaluation framework, constructing 16 reversible preprocessing pipelines—encompassing differencing and scaling techniques—based on the M4 dataset to systematically evaluate 11 models. Our findings demonstrate that preprocessing is a critical determinant of forecasting performance; when optimally configured, simple architectures achieve accuracy improvements of 27% to 87%, rendering them competitive with complex models. Furthermore, we release an open-source meta-dataset alongside this work to establish a standardized evaluation foundation for future research.

Abstract

While established literature underscores the pivotal role of preprocessing in forecasting accuracy, this stage remains largely overlooked in current research. Modern benchmarks typically resort to simple scaling, failing to account for critical transformations required to address nonstationarity, such as differencing. This omission creates a significant structural preprocessing bias that favors models with built-in data treatments while obscuring the true potential of simpler architectures. We study this effect through a preprocessing-aware benchmark that evaluates 11 forecasting models across 16 reversible preprocessing pipelines on 29,000 M4 time series. Our results identify preprocessing as a key driver of forecasting performance. Optimizing preprocessing per series yields gains of approximately 27\% to 87\% across all evaluated models, with architectures lacking internalized preprocessing experiencing the most substantial improvements. This allows simpler architectures to become highly competitive with complex, state-of-the-art models in modern forecasting benchmarks. All resources and experimental results from this benchmark are stored in a comprehensive metadataset to support future metalearning tasks.

Problem

• time series forecasting
• preprocessing bias
• benchmarking
• nonstationarity
• model evaluation

Innovation

• Preprocessing Bias
• Time Series Forecasting
• Reversible Pipelines
• Benchmarking
• Metadataset