A Review of the Long Horizon Forecasting Problem in Time Series Analysis

2025-06-15

AI Summary

A core challenge in long-term time-series forecasting is the cumulative error growth with prediction horizon. This paper formally defines Long-Horizon Forecasting (LHF) as an error propagation problem and systematically surveys 35 years of research, emphasizing deep learning advances in trend/cyclic modeling, error suppression, and feature-space construction. We propose a unified framework integrating Fourier/wavelet transforms, bandpass filtering, State Space Models (SSMs), attention masking, sparse convolutions, and multi-scale decomposition. We identify xLSTM and Triformer as architectures capable of breaking the monotonic error-growth bottleneck. Furthermore, we release the first open-source LHF model library. Empirical evaluation on the ETTm2 dataset demonstrates that both models significantly mitigate MSE escalation: on the multivariate HUFL task, they achieve an average 18.7% reduction in forecasting error.

Technology Category

Machine Learning: Time-Series/Data StreamsNatural Language Processing: (Large) Language ModelsComputer Vision: Large Vision Models

Application Category

Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphsSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsSearch and Retrieval-Augmented AI: Large language models for search

Abstract

The long horizon forecasting (LHF) problem has come up in the time series literature for over the last 35 years or so. This review covers aspects of LHF in this period and how deep learning has incorporated variants of trend, seasonality, fourier and wavelet transforms, misspecification bias reduction and bandpass filters while contributing using convolutions, residual connections, sparsity reduction, strided convolutions, attention masks, SSMs, normalization methods, low-rank approximations and gating mechanisms. We highlight time series decomposition techniques, input data preprocessing and dataset windowing schemes that improve performance. Multi-layer perceptron models, recurrent neural network hybrids, self-attention models that improve and/or address the performances of the LHF problem are described, with an emphasis on the feature space construction. Ablation studies are conducted over the ETTm2 dataset in the multivariate and univariate high useful load (HUFL) forecasting contexts, evaluated over the last 4 months of the dataset. The heatmaps of MSE averages per time step over test set series in the horizon show that there is a steady increase in the error proportionate to its length except with xLSTM and Triformer models and motivate LHF as an error propagation problem. The trained models are available here: https://bit.ly/LHFModelZoo

Problem

• Addressing error propagation in long horizon time series forecasting
• Improving forecasting performance using deep learning techniques
• Evaluating models on multivariate and univariate high useful load datasets

Innovation

• Deep learning integrates trend and seasonality transforms
• Uses convolutions, attention masks, and gating mechanisms
• Evaluates models with ablation studies on ETTm2 dataset