🤖 AI Summary
This work addresses the longstanding trade-off in spatiotemporal modeling: convolutional neural networks (CNNs) struggle to capture long-range dependencies, while Transformer-based approaches incur prohibitive computational costs. To reconcile this dilemma, the authors propose MIMO-ESP, a purely CNN-based architecture that achieves Transformer-like global receptive fields through a multi-input multi-output design, decoupled temporal modeling, and dilated convolutions. This approach effectively integrates spatial and temporal information while preserving the parallelizability and efficiency inherent to CNNs, substantially reducing computational complexity. Extensive experiments demonstrate that MIMO-ESP consistently outperforms state-of-the-art methods across three diverse benchmark datasets—covering video prediction, traffic flow forecasting, and precipitation nowcasting—thereby achieving an optimal balance between predictive accuracy and computational efficiency.
📝 Abstract
Recently, Convolutional Neural Network (CNN) or Transformer architecture based models have been proposed to overcome the limitations of Recurrent Neural Network (RNN) based models in spatiotemporal prediction. These models prevent the inefficiency of parallelization limitation due to the sequential properties and stacked error due to the recursive method, and show high performance. Novertheless, there are still some challengies. First, CNN based models have difficulty considering global information due to the local properties of the kernel, and their performance is limited. In addition, information is mixed because the time axis is combined with the channel axis of the image for processing. Models based on Transformer architecture have high complexity due to the self-attention calcuation and take a long training time. In this paper, we propose a new structure model called CNN-based Multi-In-Multi-Out model for Efficient Spatiotemporal Prediction (MIMO-ESP) to overcome these limitations. MIMO-ESP considers global information and significantly improves complexity by configuring a Transformer architecture based on CNN. In addition, it treats the time axis as an independent axis without combining it, and effectively considers spatiotemporal information together by applying dilation. This structure makes MIMO-ESP efficient and high performance. Extensive experiment results on three promising benchmark datasets which including video, traffic, and precipitation prediction tasks demonstrate that the usefulness of MIMO-ESP due to the achieved competitive efficiency while outperforming existing models. Furthermore, the ablation study results demonstrate the usefulness of the components of MIMO-ESP, emphasizing the potential of the proposed approaches.