Channel Dependence, Limited Lookback Windows, and the Simplicity of Datasets: How Biased is Time Series Forecasting?

2025-02-13

AI Summary

This paper identifies an evaluation bias in current time-series forecasting research, stemming from widespread use of simplistic datasets and fixed short lookback windows: inter-channel dependencies are only pronounced in complex data, yet state-of-the-art (SOTA) models—e.g., Crossformer—are systematically underestimated. To address this, we first rigorously characterize the bias in prevailing evaluation paradigms; second, we propose a *data-complexity-driven criterion for inter-channel dependency*, replacing static assumptions with adaptive assessment; third, we design FaCT—a lightweight model integrating dynamic lookback optimization, streamlined channel-wise attention, and computational acceleration. Empirical validation on complex benchmarks confirms the necessity of explicit channel interaction. FaCT achieves SOTA accuracy while matching TimeMixer’s inference speed. Crucially, our analysis re-establishes Crossformer as an indispensable baseline, underscoring its latent capability in modeling structured inter-channel dynamics.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Time-series forecasting research has converged to a small set of datasets and a standardized collection of evaluation scenarios. Such a standardization is to a specific extent needed for comparable research. However, the underlying assumption is, that the considered setting is a representative for the problem as a whole. In this paper, we challenge this assumption and show that the current scenario gives a strongly biased perspective on the state of time-series forecasting research. To be more detailed, we show that the current evaluation scenario is heavily biased by the simplicity of the current datasets. We furthermore emphasize, that when the lookback-window is properly tuned, current models usually do not need any information flow across channels. However, when using more complex benchmark data, the situation changes: Here, modeling channel-interactions in a sophisticated manner indeed enhances performances. Furthermore, in this complex evaluation scenario, Crossformer, a method regularly neglected as an important baseline, is the SOTA method for time series forecasting. Based on this, we present the Fast Channel-dependent Transformer (FaCT), a simplified version of Crossformer which closes the runtime gap between Crossformer and TimeMixer, leading to an efficient model for complex forecasting datasets.

Problem

• Challenges bias in time-series forecasting evaluation
• Demonstrates dataset simplicity skews model performance
• Introduces FaCT for efficient complex dataset forecasting

Innovation

• Simplified Crossformer
• Fast Channel-dependent Transformer
• Enhanced channel-interaction modeling