🤖 AI Summary
This study addresses the limitations of rule-based model selection in time series forecasting, which often fails due to variations in data-generating mechanisms. The authors propose a descriptor framework grounded in measurable characteristics—such as trend strength, seasonality, noise level, and temporal dependence—and systematically evaluate the efficacy of static selection rules across diverse real-world datasets. Their analysis reveals, for the first time, that static descriptors are insufficient for reliably predicting model performance: model selection proves highly context-dependent and unstable. Notably, under noisy conditions or mixed generative mechanisms, recommended models frequently diverge substantially from the true best-performing ones, and model rankings exhibit pronounced sensitivity to both data characteristics and forecast horizons.
📝 Abstract
Time series forecasting models often exhibit inconsistent performance across datasets with varying statistical and structural properties. Despite the wide range of available forecasting techniques, it remains unclear whether model selection can be reliably guided by simple data characteristics. This paper investigates why rule-based model selection fails in time series forecasting by analyzing the relationship between data-regime descriptors and model performance. A descriptor-based framework is introduced to characterize time series using measurable properties, including trend strength, seasonality, noise level, and temporal dependence. Based on these descriptors, a rule-based selection mechanism is formulated to map data regimes to candidate forecasting models. The approach is evaluated on multiple real-world datasets across different domains and forecasting horizons. The results show that rule-based model selection achieves low accuracy, with correct model identification occurring in only a small fraction of cases. Significant discrepancies are observed between recommended and empirically optimal models, particularly in noisy and mixed regimes. Further analysis reveals that model performance is highly sensitive to both dataset characteristics and forecasting horizon, resulting in substantial ranking instability across scenarios. These findings explain why simple heuristic rules fail to generalize and demonstrate that forecasting performance cannot be reliably predicted using static, descriptor-based approaches. This study provides empirical evidence that model selection in time series forecasting is inherently context-dependent and highlights the need for more adaptive, data-driven strategies.