MixBench-TS: A Multivariate Time Series Forecasting Benchmark Where Channel Mixing Pays Off

2026-09-26

AI Summary

This study addresses the deficiency of existing multivariate time series forecasting benchmarks in validating cross-channel coupling, which leads to distorted model evaluation. We propose the first Channel Dependency (CD) Gain metric, integrating lagged mutual information, Granger causality, and transfer entropy, to reveal the insufficient coupling inherent in standard datasets. Building upon this insight, we construct MixBench, a real-world benchmark characterized by high inter-channel coupling, and systematically evaluate mainstream multivariate forecasting models. Experimental results demonstrate that on MixBench, channel-mixing models significantly outperform channel-independent approaches, confirming the critical value of cross-channel coupling for accurate forecasting. This work establishes a more rigorous evaluation paradigm for multivariate time series forecasting.

Abstract

Multivariate Time Series Forecasting (MTSF) models that mix information across channels assume that the past of one channel carries information about the future of another. Yet they are evaluated on a small fixed set of standard datasets whose cross-channel structure is rarely examined. We ask two questions:"How can we reliably measure lagged, non-linear, and joint coupling in MTSF datasets?"and"Do the standard datasets actually have such coupling?"To answer the first, we test four candidate measures on synthetic datasets with planted ground-truth coupling: Granger Causality (GC), Transfer Entropy (TE), lagged Mutual Information (MI), and the CD gain, a model-based measure we introduce that compares a channel-dependent (CD) model to its channel-independent (CI) variant. Only lagged MI and the CD gain recover every planted coupling. For the second question, the answer is a definite no, as the standard datasets have a median of only 23% lagged-coupled channel pairs and a median CD gain of -4.9%, compared to 78% and +4.6% on chaotic ODE systems. We therefore propose MixBench-TS, a benchmark of 10 real-world datasets with a median of 55.5% lagged-coupled pairs and a median CD gain of +1.7%. Across six state-of-the-art models tuned under one protocol, CI models win on 10/10 (MSE) and 8/10 (MAE) standard datasets, but on only 3/10 and 2/10 MixBench-TS datasets. We recommend using our benchmark for evaluating new CD models. Moreover, we propose profiling new datasets with lagged MI and the CD gain before using them to evaluate multivariate models. Code and data are available at https://anonymous.4open.science/r/mixbench-ts-B027.

Problem

• Multivariate Time Series Forecasting
• Cross-channel Coupling
• Benchmark Evaluation
• Channel Mixing

Innovation

• Multivariate Time Series Forecasting
• Channel Mixing
• CD Gain
• Lagged Mutual Information
• MixBench-TS